Jul 02 13:43:02.442303 ip-10-0-137-226 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 02 13:43:02.442317 ip-10-0-137-226 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 02 13:43:02.442327 ip-10-0-137-226 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 02 13:43:02.442633 ip-10-0-137-226 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 02 13:43:12.491423 ip-10-0-137-226 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 02 13:43:12.491440 ip-10-0-137-226 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 543d9b04ce624cc8a3c9201c645ee8f9 --
Jul 02 13:45:13.355150 ip-10-0-137-226 systemd[1]: Starting Kubernetes Kubelet...
Jul 02 13:45:13.729541 ip-10-0-137-226 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 13:45:13.729541 ip-10-0-137-226 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 02 13:45:13.729541 ip-10-0-137-226 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 13:45:13.729541 ip-10-0-137-226 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 02 13:45:13.729541 ip-10-0-137-226 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 13:45:13.731400 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.731274    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 02 13:45:13.735647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735626    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:45:13.735647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735646    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735649    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735653    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735657    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735660    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735664    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735667    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735671    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735676    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735680    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735684    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735687    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735690    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735693    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735696    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735698    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735701    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735704    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735707    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:45:13.735721 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735709    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735712    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735715    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735717    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735720    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735722    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735725    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735728    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735732    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735734    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735737    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735740    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735742    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735745    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735748    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735750    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735753    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735755    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735758    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735761    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:45:13.736204 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735763    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735766    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735768    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735771    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735773    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735775    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735778    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735781    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735783    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735786    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735789    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735791    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735794    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735797    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735800    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735802    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735805    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735807    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735810    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:45:13.736680 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735819    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735822    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735824    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735827    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735830    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735832    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735836    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735840    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735843    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735858    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735861    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735865    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735867    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735870    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735873    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735875    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735879    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735882    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735884    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735887    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:45:13.737167 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735890    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735893    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735895    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735898    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735900    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735903    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735908    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735912    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735915    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735918    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735920    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735923    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735926    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735928    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735931    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735933    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735936    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735938    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735941    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.735944    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:45:13.737665 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736677    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736683    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736686    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736688    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736692    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736695    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736698    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736700    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736703    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736706    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736708    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736711    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736714    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736717    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736720    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736723    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736725    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736727    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736730    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736733    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:45:13.738193 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736736    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736738    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736741    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736743    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736746    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736748    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736752    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736755    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736757    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736760    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736762    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736765    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736768    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736770    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736773    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736775    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736778    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736781    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736785    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736787    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:45:13.738840 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736790    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736792    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736797    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736801    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736804    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736807    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736810    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736813    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736816    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736819    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736821    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736824    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736827    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736830    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736833    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736836    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736838    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736841    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736844    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:45:13.739607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736861    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736865    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736869    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736873    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736877    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736881    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736884    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736887    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736890    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736893    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736895    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736900    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736904    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736906    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736909    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736912    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736915    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736917    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736921    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:45:13.740127 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736924    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736927    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736929    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736932    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736934    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736937    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736939    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736942    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736945    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736949    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736952    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736955    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736958    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736960    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736963    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736966    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736968    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736971    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736974    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:45:13.740607 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736976    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.736979    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737361    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737366    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737369    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737372    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737375    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737378    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737380    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737384    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737386    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737389    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737392    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737395    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737397    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737400    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737402    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737405    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737407    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737409    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:45:13.741082 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737412    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737414    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737417    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737419    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737422    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737425    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737427    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737430    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737433    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737436    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737438    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737441    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737443    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737446    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737448    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737451    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737454    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737457    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737460    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737463    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:45:13.741580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737465    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737468    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737470    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737473    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737475    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737486    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737489    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737492    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737494    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737497    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737499    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737502    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737507    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737510    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737513    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737516    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737519    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737521    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737524    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:45:13.742070 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737527    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737529    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737532    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737534    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737537    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737539    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737543    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737545    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737549    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737551    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737554    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737557    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737560    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737562    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737567    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737571    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737574    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737578    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737581    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:45:13.742521 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737584    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737587    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737590    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737593    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737595    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737598    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737600    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737603    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737605    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737607    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737610    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737613    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737615    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737618    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737620    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737623    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737625    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737628    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737630    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737633    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:45:13.743005 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737635    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737638    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.737640    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737723    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737733    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737744    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737752    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737756    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737760    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737765    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737769    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737772    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737776    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737779    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737783    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737786    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737789    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737792    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737795    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737798    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737801    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737805    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737808    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737811    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 02 13:45:13.743511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737814    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737817    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737821    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737824    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737827    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737830    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737834    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737837    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737840    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737843    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737857    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737864    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737867    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737870    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737874    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737877    2567 flags.go:64] FLAG: --enable-server="true"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737880    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737885    2567 flags.go:64] FLAG: --event-burst="100"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737888    2567 flags.go:64] FLAG: --event-qps="50"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737891    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737894    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737897    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737901    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737904    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737907    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 02 13:45:13.744109 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737910    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737913    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737916    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737919    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737921    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737924    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737927    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737930    2567 flags.go:64] FLAG: --feature-gates=""
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737934    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737937    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737940    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737943    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737947    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737950    2567 flags.go:64] FLAG: --help="false"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737952    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-137-226.ec2.internal"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737956    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737959    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737962    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737967    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737970    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737973    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737976    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737979    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 02 13:45:13.744714 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737983    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737986    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737989    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737992    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737995    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.737998    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738001    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738004    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738007    2567 flags.go:64] FLAG: --lock-file=""
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738010    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738012    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738015    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738021    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738024    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738027    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738030    2567 flags.go:64] FLAG: --logging-format="text"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738032    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738036    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738038    2567 flags.go:64] FLAG: --manifest-url=""
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738041    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738046    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738049    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738053    2567 flags.go:64] FLAG: --max-pods="110"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738056    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738060    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 02 13:45:13.745341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738063    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738066    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738070    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738073    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738076    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738084    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738087    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738090    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738094    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738097    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738104    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738107    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738111    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738114    2567 flags.go:64] FLAG: --port="10250"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738118    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738121    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09ed270cb8c2834e0"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738124    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738127    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738130    2567 flags.go:64] FLAG: --register-node="true"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738133    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738142    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738145    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738147    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738150    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738154    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 02 13:45:13.746012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738157    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738160    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738163    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738166    2567 flags.go:64] FLAG: --runonce="false"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738169    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738172    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738176    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738179    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738182    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738185    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738189    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738192    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738196    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738199    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738202    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738205    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738209    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738212    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738215    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738221    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738223    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738227    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738231    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738233    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738236    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 02 13:45:13.746637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738239    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738242    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738245    2567 flags.go:64] FLAG: --v="2"
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738249    2567 flags.go:64] FLAG: --version="false"
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738255    2567 flags.go:64] FLAG: --vmodule=""
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738259    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.738262    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738726    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738739    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738744    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738750    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738754    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738759    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738766    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738771    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738776    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738788    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738793    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738798    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738805    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738812    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:45:13.747347 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738823    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738828    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738832    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738864    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738869    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738874    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738878    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738887    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738891    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738895    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738900    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738904    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738908    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738912    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738916    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738920    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738924    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738928    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738932    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:45:13.747876 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738937    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738946    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738950    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738954    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738958    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738964    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738968    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738972    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738976    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738981    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738985    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738989    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.738993    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739002    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739007    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739012    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739016    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739021    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739025    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:45:13.748343 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739029    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739034    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739039    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739043    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739047    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739052    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739056    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739065    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739069    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739073    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739078    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739082    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739086    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739091    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739095    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739099    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739103    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739109    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739114    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739123    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:45:13.748890 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739127    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739131    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739136    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739140    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739144    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739148    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739155    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739160    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739164    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739169    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739174    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739184    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739189    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739194    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739198    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739203    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739208    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739212    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739217    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739221    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:45:13.749773 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739225    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739229    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739234    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739238    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739247    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739251    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.739255    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.739718    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.746468    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.746484    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.747158    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.747166    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747225    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747231    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747235    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:45:13.750373 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747238    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747241    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747244    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747247    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747250    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747253    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747255    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747258    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747261    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747264    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747267    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747269    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747272    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747275    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747277    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747281    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747285    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747288    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747291    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747295    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:45:13.750913 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747297    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747300    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747303    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747305    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747308    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747311    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747313    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747317    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747322    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747325    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747328    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747331    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747334    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747336    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747339    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747341    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747344    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747347    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747349    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:45:13.751429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747352    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747356    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747358    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747362    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747366    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747370    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747372    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747375    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747379    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747381    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747384    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747386    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747389    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747392    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747395    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747397    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747400    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747402    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747405    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:45:13.751941 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747408    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747410    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747413    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747416    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747419    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747421    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747424    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747426    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747429    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747432    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747434    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747437    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747439    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747442    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747444    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747447    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747450    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747452    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747454    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747457    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:45:13.752429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747459    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747462    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747465    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747467    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747470    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747472    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747475    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747477    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747480    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747483    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747485    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747488    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747491    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747493    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747496    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747499    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747501    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:45:13.753059 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747503    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.747508    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747622    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747626    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747629    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747632    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747635    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747638    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747641    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747644    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747647    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747650    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747652    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747655    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747658    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:45:13.753508 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747661    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747664    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747666    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747669    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747671    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747675    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747679    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747682    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747685    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747687    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747690    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747693    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747695    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747698    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747701    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747703    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747706    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747708    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747711    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747713    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:45:13.753909 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747716    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747719    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747721    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747724    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747726    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747729    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747731    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747733    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747736    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747738    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747741    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747744    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747747    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747750    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747753    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747755    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747758    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747760    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747763    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:45:13.754647 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747766    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747770    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747773    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747776    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747778    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747781    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747784    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747787    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747790    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747792    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747796    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747799    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747802    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747805    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747808    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747810    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747813    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747815    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747818    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:45:13.755427 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747821    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747823    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747826    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747829    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747831    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747834    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747836    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747839    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747842    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747844    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747862    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747865    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747871    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747873    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747876    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747879    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747882    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747885    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747887    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:45:13.756232 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747890    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747893    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747895    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747898    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747900    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747903    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747905    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747908    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:13.747910    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.747916    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.748039    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.750493    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.751882    2567 server.go:1019] "Starting client certificate rotation"
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.751992    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 13:45:13.756949 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.752024    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 13:45:13.757319 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.754755    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 13:45:13.757319 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.756578    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 13:45:13.764794 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.764778    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 02 13:45:13.766739 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.766724    2567 log.go:25] "Validated CRI v1 image API"
Jul 02 13:45:13.768018 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.767921    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 02 13:45:13.770430 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.770393    2567 fs.go:135] Filesystem UUIDs: map[2f1e01ee-dbbc-4e9a-bfdf-a5000cc69ab3:/dev/nvme0n1p4 711570be-b71b-4d41-8795-692b74a0adad:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 02 13:45:13.770497 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.770429    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 02 13:45:13.776711 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.776597    2567 manager.go:216] Machine: {Timestamp:2026-07-02 13:45:13.775458132 +0000 UTC m=+0.308214783 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100365 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec231eb413a4a669d2d3eac3e9a441c7 SystemUUID:ec231eb4-13a4-a669-d2d3-eac3e9a441c7 BootID:543d9b04-ce62-4cc8-a3c9-201c645ee8f9 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5e:10:b8:a8:b7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5e:10:b8:a8:b7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:10:61:5b:a4:a3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 02 13:45:13.776711 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.776701    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 02 13:45:13.776845 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.776785    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 02 13:45:13.779528 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.779502    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 02 13:45:13.779663 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.779531    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-226.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 02 13:45:13.780145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.780135    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 02 13:45:13.780182 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.780146    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 02 13:45:13.780182 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.780154    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 13:45:13.780729 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.780719    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 13:45:13.781512 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.781502    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 02 13:45:13.784071 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.784060    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 13:45:13.784190 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.784181    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 02 13:45:13.784946 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.784929    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 02 13:45:13.784997 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.784949    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 02 13:45:13.784997 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.784967    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 02 13:45:13.784997 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.784977    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 02 13:45:13.784997 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.784986    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 02 13:45:13.785335 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.785325    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 13:45:13.785369 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.785344    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 13:45:13.787355 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.787338    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 02 13:45:13.788639 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.788626    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 02 13:45:13.788682 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.788656    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 02 13:45:13.789262 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789248    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 02 13:45:13.789322 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789270    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 02 13:45:13.789322 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789279    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 02 13:45:13.789322 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789288    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 02 13:45:13.789322 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789300    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 02 13:45:13.789322 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789306    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 02 13:45:13.789322 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789316    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 02 13:45:13.789322 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789322    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 02 13:45:13.789545 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789328    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 02 13:45:13.789545 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789334    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 02 13:45:13.789545 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789348    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 02 13:45:13.789545 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.789354    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 02 13:45:13.790132 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.790120    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 02 13:45:13.790132 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.790133    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 02 13:45:13.793454 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.793441    2567 server.go:1268] "Started kubelet"
Jul 02 13:45:13.793586 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.793530    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 02 13:45:13.793586 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.793550    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 02 13:45:13.793689 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.793610    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 02 13:45:13.794409 ip-10-0-137-226 systemd[1]: Started Kubernetes Kubelet.
Jul 02 13:45:13.794699 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.794681    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 02 13:45:13.794789 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.794712    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 02 13:45:13.798915 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.798888    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 02 13:45:13.800020 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.800005    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 02 13:45:13.800071 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.800020    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 02 13:45:13.800120 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.800087    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
Jul 02 13:45:13.800507 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.800487    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 02 13:45:13.800507 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.800489    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 02 13:45:13.800648 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.800511    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 02 13:45:13.800768 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.800753    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 02 13:45:13.800824 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.800771    2567 factory.go:55] Registering systemd factory
Jul 02 13:45:13.800824 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.800780    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 02 13:45:13.801018 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.800998    2567 factory.go:153] Registering CRI-O factory
Jul 02 13:45:13.801018 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.801019    2567 factory.go:223] Registration of the crio container factory successfully
Jul 02 13:45:13.801156 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.801041    2567 factory.go:103] Registering Raw factory
Jul 02 13:45:13.801156 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.801054    2567 manager.go:1196] Started watching for new ooms in manager
Jul 02 13:45:13.801454 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.801440    2567 manager.go:349] Starting recovery of all containers
Jul 02 13:45:13.801600 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.801587    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 02 13:45:13.801600 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.801598    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 02 13:45:13.813646 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.813620    2567 manager.go:354] Recovery completed
Jul 02 13:45:13.815112 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.815084    2567 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 02 13:45:13.818373 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.818360    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:45:13.820222 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.820205    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:45:13.820274 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.820239    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:45:13.820274 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.820253    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:45:13.820764 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.820750    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 02 13:45:13.820811 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.820764    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 02 13:45:13.820811 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.820784    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 13:45:13.820811 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.820754    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:45:13.821557 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.821543    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:45:13.821608 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.821570    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:45:13.821608 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.821580    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:45:13.821843 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.821825    2567 policy_none.go:49] "None policy: Start"
Jul 02 13:45:13.821898 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.821879    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 02 13:45:13.821898 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.821891    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 02 13:45:13.822773 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.822750    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 02 13:45:13.822871 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.822816    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 02 13:45:13.822871 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.822830    2567 policy_none.go:47] "Start"
Jul 02 13:45:13.823062 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.823047    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-226.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 02 13:45:13.823776 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.822908    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf9903569ba  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.79341561 +0000 UTC m=+0.326172242,LastTimestamp:2026-07-02 13:45:13.79341561 +0000 UTC m=+0.326172242,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.823909 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.823889    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-226.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 02 13:45:13.823980 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.823917    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-226.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 02 13:45:13.831174 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.831001    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ce77f8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.82022348 +0000 UTC m=+0.352980112,LastTimestamp:2026-07-02 13:45:13.82022348 +0000 UTC m=+0.352980112,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.843603 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.843500    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ced1b8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820246456 +0000 UTC m=+0.353003085,LastTimestamp:2026-07-02 13:45:13.820246456 +0000 UTC m=+0.353003085,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.847205 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.847130    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991cefbc9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820257225 +0000 UTC m=+0.353013856,LastTimestamp:2026-07-02 13:45:13.820257225 +0000 UTC m=+0.353013856,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.849514 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.849499    2567 manager.go:337] "Starting Device Plugin manager"
Jul 02 13:45:13.849566 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.849535    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 02 13:45:13.849566 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.849546    2567 server.go:87] "Starting device plugin registration server"
Jul 02 13:45:13.849807 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.849796    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 02 13:45:13.849872 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.849809    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 02 13:45:13.849947 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.849930    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 02 13:45:13.850037 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.850016    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 02 13:45:13.850037 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.850031    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 02 13:45:13.850489 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.850473    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 02 13:45:13.850549 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.850514    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-226.ec2.internal\" not found"
Jul 02 13:45:13.868631 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.868596    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 02 13:45:13.875420 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.869826    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 02 13:45:13.875420 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.869879    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 02 13:45:13.875420 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.869902    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 02 13:45:13.875420 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.869933    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 02 13:45:13.875420 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.873726    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 02 13:45:13.875420 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.873748    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991ce77f8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ce77f8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.82022348 +0000 UTC m=+0.352980112,LastTimestamp:2026-07-02 13:45:13.821557894 +0000 UTC m=+0.354314525,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.875647 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.875519    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 13:45:13.889559 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.889491    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991ced1b8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ced1b8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820246456 +0000 UTC m=+0.353003085,LastTimestamp:2026-07-02 13:45:13.821574874 +0000 UTC m=+0.354331505,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.895724 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.895662    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991cefbc9\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991cefbc9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820257225 +0000 UTC m=+0.353013856,LastTimestamp:2026-07-02 13:45:13.821584422 +0000 UTC m=+0.354341053,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.907662 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.907574    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf993af2662  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.85172541 +0000 UTC m=+0.384482031,LastTimestamp:2026-07-02 13:45:13.85172541 +0000 UTC m=+0.384482031,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.950129 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.950095    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:45:13.951106 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.951088    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:45:13.951171 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.951121    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:45:13.951171 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.951132    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:45:13.951171 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.951153    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-226.ec2.internal"
Jul 02 13:45:13.959557 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.959463    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991ce77f8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ce77f8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.82022348 +0000 UTC m=+0.352980112,LastTimestamp:2026-07-02 13:45:13.951106586 +0000 UTC m=+0.483863218,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.970282 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.970255    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"]
Jul 02 13:45:13.970389 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.970333    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:45:13.971681 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.971660    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:45:13.971806 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.971694    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:45:13.971806 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.971706    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:45:13.973174 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.973160    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:45:13.973277 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.973262    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:13.973324 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.973311    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:45:13.973957 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.973942    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:45:13.974049 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.973967    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:45:13.974049 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.973976    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:45:13.974454 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.974440    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:45:13.974508 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.974466    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:45:13.974508 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.974477    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:45:13.975675 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.975648    2567 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-137-226.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-137-226.ec2.internal"
Jul 02 13:45:13.975783 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.975664    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991ced1b8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ced1b8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820246456 +0000 UTC m=+0.353003085,LastTimestamp:2026-07-02 13:45:13.951125595 +0000 UTC m=+0.483882225,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.977662 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.977642    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:13.977757 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.977679    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:45:13.978434 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.978419    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:45:13.978520 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.978446    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:45:13.978520 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:13.978460    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:45:13.978829 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.978768    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991cefbc9\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991cefbc9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820257225 +0000 UTC m=+0.353013856,LastTimestamp:2026-07-02 13:45:13.951136095 +0000 UTC m=+0.483892725,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.989554 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.989468    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991ce77f8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ce77f8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.82022348 +0000 UTC m=+0.352980112,LastTimestamp:2026-07-02 13:45:13.971678024 +0000 UTC m=+0.504434655,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.992138 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.992071    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991ced1b8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ced1b8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820246456 +0000 UTC m=+0.353003085,LastTimestamp:2026-07-02 13:45:13.971701045 +0000 UTC m=+0.504457675,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:13.998939 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:13.998865    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991cefbc9\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991cefbc9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820257225 +0000 UTC m=+0.353013856,LastTimestamp:2026-07-02 13:45:13.971711027 +0000 UTC m=+0.504467659,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:14.002704 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.002685    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa3e9918aa0232d73fb5a4effb0efd9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"fa3e9918aa0232d73fb5a4effb0efd9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.002766 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.002713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa3e9918aa0232d73fb5a4effb0efd9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"fa3e9918aa0232d73fb5a4effb0efd9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.002766 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.002731    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f18bf94e33493ecb15787b6bbd7e24d0-config\") pod \"kube-apiserver-proxy-ip-10-0-137-226.ec2.internal\" (UID: \"f18bf94e33493ecb15787b6bbd7e24d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.005666 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.005649    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-226.ec2.internal\" not found" node="ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.010306 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.010291    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-226.ec2.internal\" not found" node="ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.018913 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.018828    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991ce77f8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ce77f8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.82022348 +0000 UTC m=+0.352980112,LastTimestamp:2026-07-02 13:45:13.973954417 +0000 UTC m=+0.506711048,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:14.027590 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.027510    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991ced1b8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ced1b8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820246456 +0000 UTC m=+0.353003085,LastTimestamp:2026-07-02 13:45:13.973971271 +0000 UTC m=+0.506727901,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:14.027712 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.027593    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-226.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms"
Jul 02 13:45:14.030576 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.030513    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991cefbc9\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991cefbc9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820257225 +0000 UTC m=+0.353013856,LastTimestamp:2026-07-02 13:45:13.973979661 +0000 UTC m=+0.506736291,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:14.053113 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.053032    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991ce77f8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ce77f8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.82022348 +0000 UTC m=+0.352980112,LastTimestamp:2026-07-02 13:45:13.974453302 +0000 UTC m=+0.507209933,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:14.062879 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.062781    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991ced1b8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ced1b8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820246456 +0000 UTC m=+0.353003085,LastTimestamp:2026-07-02 13:45:13.974472183 +0000 UTC m=+0.507228814,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:14.085086 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.084988    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991cefbc9\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991cefbc9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.820257225 +0000 UTC m=+0.353013856,LastTimestamp:2026-07-02 13:45:13.974481418 +0000 UTC m=+0.507238049,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:14.085819 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.085800    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d6k8g"
Jul 02 13:45:14.102900 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.102876    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa3e9918aa0232d73fb5a4effb0efd9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"fa3e9918aa0232d73fb5a4effb0efd9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.103019 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.102907    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa3e9918aa0232d73fb5a4effb0efd9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"fa3e9918aa0232d73fb5a4effb0efd9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.103019 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.102923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f18bf94e33493ecb15787b6bbd7e24d0-config\") pod \"kube-apiserver-proxy-ip-10-0-137-226.ec2.internal\" (UID: \"f18bf94e33493ecb15787b6bbd7e24d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.103019 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.102966    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f18bf94e33493ecb15787b6bbd7e24d0-config\") pod \"kube-apiserver-proxy-ip-10-0-137-226.ec2.internal\" (UID: \"f18bf94e33493ecb15787b6bbd7e24d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.103019 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.102993    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa3e9918aa0232d73fb5a4effb0efd9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"fa3e9918aa0232d73fb5a4effb0efd9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.103019 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.102996    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fa3e9918aa0232d73fb5a4effb0efd9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal\" (UID: \"fa3e9918aa0232d73fb5a4effb0efd9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.104709 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.104635    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-226.ec2.internal.18be7cf991ce77f8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-226.ec2.internal.18be7cf991ce77f8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-226.ec2.internal,UID:ip-10-0-137-226.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-226.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-226.ec2.internal,},FirstTimestamp:2026-07-02 13:45:13.82022348 +0000 UTC m=+0.352980112,LastTimestamp:2026-07-02 13:45:13.978434788 +0000 UTC m=+0.511191426,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-226.ec2.internal,}"
Jul 02 13:45:14.105122 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.105104    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d6k8g"
Jul 02 13:45:14.176774 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.176745    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:45:14.177840 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.177820    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:45:14.177986 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.177872    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:45:14.177986 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.177886    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:45:14.177986 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.177921    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.202587 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.202561    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.202701 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.202591    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-226.ec2.internal\": node \"ip-10-0-137-226.ec2.internal\" not found"
Jul 02 13:45:14.267586 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.267505    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
Jul 02 13:45:14.308097 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.308051    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.314081 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.314059    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.368651 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.368606    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
Jul 02 13:45:14.469469 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.469440    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
Jul 02 13:45:14.569965 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.569877    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-226.ec2.internal\" not found"
Jul 02 13:45:14.662062 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.662038    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:45:14.700452 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.700423    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.716005 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.715979    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 13:45:14.717720 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.717704    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal"
Jul 02 13:45:14.725451 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.725430    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:45:14.736264 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.736244    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 13:45:14.751448 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.751410    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 02 13:45:14.751591 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.751569    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 02 13:45:14.785761 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.785732    2567 apiserver.go:52] "Watching apiserver"
Jul 02 13:45:14.792518 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.792495    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 02 13:45:14.792902 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.792878    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-whrk8","kube-system/global-pull-secret-syncer-jbfz7","kube-system/konnectivity-agent-8sdgz","openshift-image-registry/node-ca-f8rvf","openshift-network-diagnostics/network-check-target-bt4j2","openshift-network-operator/iptables-alerter-qzt8s","openshift-ovn-kubernetes/ovnkube-node-xtms2","kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f","openshift-cluster-node-tuning-operator/tuned-jrhsd","openshift-dns/node-resolver-zqz2f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal","openshift-multus/multus-additional-cni-plugins-n88db","openshift-multus/multus-md66t"]
Jul 02 13:45:14.794335 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.794313    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:14.794400 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.794386    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:14.795606 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.795585    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:14.795686 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.795661    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:14.796738 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.796720    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:14.798312 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.798297    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f8rvf"
Jul 02 13:45:14.799587 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.799569    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:14.799678 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.799649    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:14.800112 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.800097    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 02 13:45:14.801064 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.800637    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 02 13:45:14.801064 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.800881    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 02 13:45:14.801064 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.800956    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kx89g\""
Jul 02 13:45:14.801400 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.801383    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qzt8s"
Jul 02 13:45:14.803492 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.803434    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.803580 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.803541    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 02 13:45:14.803644 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.803542    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 02 13:45:14.805086 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.805070    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:14.806452 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.806425    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.807660 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.807642    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 02 13:45:14.807862 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.807838    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zqz2f"
Jul 02 13:45:14.808381 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808135    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 02 13:45:14.808381 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808245    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 02 13:45:14.808381 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808359    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:14.808577 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808396    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:14.808577 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808422    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a44edce6-3bb4-47ec-ac53-7724912c6277-agent-certs\") pod \"konnectivity-agent-8sdgz\" (UID: \"a44edce6-3bb4-47ec-ac53-7724912c6277\") " pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:14.808577 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808444    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/41ed71d6-1988-4c22-b3af-36711ebeef44-host-slash\") pod \"iptables-alerter-qzt8s\" (UID: \"41ed71d6-1988-4c22-b3af-36711ebeef44\") " pod="openshift-network-operator/iptables-alerter-qzt8s"
Jul 02 13:45:14.808577 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808467    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49ea922e-112c-480c-9a70-7992f0e29149-kubelet-config\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:14.808577 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808490    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a44edce6-3bb4-47ec-ac53-7724912c6277-konnectivity-ca\") pod \"konnectivity-agent-8sdgz\" (UID: \"a44edce6-3bb4-47ec-ac53-7724912c6277\") " pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:14.808577 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808512    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/110466f9-e7b0-4a07-8a64-126c9ebaf9c1-host\") pod \"node-ca-f8rvf\" (UID: \"110466f9-e7b0-4a07-8a64-126c9ebaf9c1\") " pod="openshift-image-registry/node-ca-f8rvf"
Jul 02 13:45:14.808577 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808523    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tf7js\""
Jul 02 13:45:14.808577 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808535    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tlcc\" (UniqueName: \"kubernetes.io/projected/110466f9-e7b0-4a07-8a64-126c9ebaf9c1-kube-api-access-6tlcc\") pod \"node-ca-f8rvf\" (UID: \"110466f9-e7b0-4a07-8a64-126c9ebaf9c1\") " pod="openshift-image-registry/node-ca-f8rvf"
Jul 02 13:45:14.808577 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808577    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/41ed71d6-1988-4c22-b3af-36711ebeef44-iptables-alerter-script\") pod \"iptables-alerter-qzt8s\" (UID: \"41ed71d6-1988-4c22-b3af-36711ebeef44\") " pod="openshift-network-operator/iptables-alerter-qzt8s"
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808608    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46ldx\" (UniqueName: \"kubernetes.io/projected/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-kube-api-access-46ldx\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808632    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49ea922e-112c-480c-9a70-7992f0e29149-dbus\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808651    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808700    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v77np\""
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808786    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808654    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44gdt\" (UniqueName: \"kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt\") pod \"network-check-target-bt4j2\" (UID: \"afddd3e6-b476-40e8-95d8-2b2332236c91\") " pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808875    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvdkf\" (UniqueName: \"kubernetes.io/projected/41ed71d6-1988-4c22-b3af-36711ebeef44-kube-api-access-tvdkf\") pod \"iptables-alerter-qzt8s\" (UID: \"41ed71d6-1988-4c22-b3af-36711ebeef44\") " pod="openshift-network-operator/iptables-alerter-qzt8s"
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808890    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dhs4w\""
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808905    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/110466f9-e7b0-4a07-8a64-126c9ebaf9c1-serviceca\") pod \"node-ca-f8rvf\" (UID: \"110466f9-e7b0-4a07-8a64-126c9ebaf9c1\") " pod="openshift-image-registry/node-ca-f8rvf"
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808982    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808992    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 02 13:45:14.809015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.808996    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 02 13:45:14.809542 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.809057    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-76s8s\""
Jul 02 13:45:14.809542 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.809168    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 02 13:45:14.809542 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.809197    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 02 13:45:14.809542 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.809222    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 02 13:45:14.809542 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.809380    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 02 13:45:14.809542 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.809442    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 02 13:45:14.809819 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.809597    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6bc72\""
Jul 02 13:45:14.809896 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.809835    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-66zx7\""
Jul 02 13:45:14.809980 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.809964    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 02 13:45:14.810104 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.810088    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 02 13:45:14.810205 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.810136    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 02 13:45:14.810393 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.810379    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:14.811557 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.811543    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 02 13:45:14.811840 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.811825    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.813547 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.813529    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 02 13:45:14.813547 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.813541    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nhxgh\""
Jul 02 13:45:14.813678 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.813528    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 02 13:45:14.813678 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.813589    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 02 13:45:14.814667 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.814651    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 02 13:45:14.814757 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.814694    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 02 13:45:14.814817 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.814694    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6dxrd\""
Jul 02 13:45:14.818568 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.818546    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 13:45:14.819065 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.819053    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 02 13:45:14.831172 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:14.831097    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3e9918aa0232d73fb5a4effb0efd9d.slice/crio-4396f1193e3f7f396aa85117a7093060827cf177ebd0d670c6f2007b4e60beac WatchSource:0}: Error finding container 4396f1193e3f7f396aa85117a7093060827cf177ebd0d670c6f2007b4e60beac: Status 404 returned error can't find the container with id 4396f1193e3f7f396aa85117a7093060827cf177ebd0d670c6f2007b4e60beac
Jul 02 13:45:14.831314 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:14.831299    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf18bf94e33493ecb15787b6bbd7e24d0.slice/crio-5b616a1469d54a2292419d03da663259c7fccb993e73a15728e2fe8e821468b7 WatchSource:0}: Error finding container 5b616a1469d54a2292419d03da663259c7fccb993e73a15728e2fe8e821468b7: Status 404 returned error can't find the container with id 5b616a1469d54a2292419d03da663259c7fccb993e73a15728e2fe8e821468b7
Jul 02 13:45:14.837614 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.837593    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 13:45:14.873000 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.872951    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal" event={"ID":"f18bf94e33493ecb15787b6bbd7e24d0","Type":"ContainerStarted","Data":"5b616a1469d54a2292419d03da663259c7fccb993e73a15728e2fe8e821468b7"}
Jul 02 13:45:14.874962 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.874475    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal" event={"ID":"fa3e9918aa0232d73fb5a4effb0efd9d","Type":"ContainerStarted","Data":"4396f1193e3f7f396aa85117a7093060827cf177ebd0d670c6f2007b4e60beac"}
Jul 02 13:45:14.874962 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.874560    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pbtl6"
Jul 02 13:45:14.884858 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.884825    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pbtl6"
Jul 02 13:45:14.901771 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.901738    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 02 13:45:14.909385 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909351    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:14.909489 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909398    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-tmp\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.909489 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909425    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-cnibin\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.909489 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909469    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-run-k8s-cni-cncf-io\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.909586 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909523    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tlcc\" (UniqueName: \"kubernetes.io/projected/110466f9-e7b0-4a07-8a64-126c9ebaf9c1-kube-api-access-6tlcc\") pod \"node-ca-f8rvf\" (UID: \"110466f9-e7b0-4a07-8a64-126c9ebaf9c1\") " pod="openshift-image-registry/node-ca-f8rvf"
Jul 02 13:45:14.909618 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909557    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-kubelet\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.909652 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909628    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-slash\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.909652 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909646    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-os-release\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:14.909730 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909671    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtgvs\" (UniqueName: \"kubernetes.io/projected/f24875be-dc41-4b03-84e6-e00f7c5d9df0-kube-api-access-mtgvs\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:14.909730 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909699    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46ldx\" (UniqueName: \"kubernetes.io/projected/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-kube-api-access-46ldx\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:14.909730 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909717    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvdkf\" (UniqueName: \"kubernetes.io/projected/41ed71d6-1988-4c22-b3af-36711ebeef44-kube-api-access-tvdkf\") pod \"iptables-alerter-qzt8s\" (UID: \"41ed71d6-1988-4c22-b3af-36711ebeef44\") " pod="openshift-network-operator/iptables-alerter-qzt8s"
Jul 02 13:45:14.909882 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909732    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-var-lib-openvswitch\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.909882 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909746    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-sysconfig\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.909882 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909768    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-host\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.909882 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909791    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f24875be-dc41-4b03-84e6-e00f7c5d9df0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:14.909882 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909811    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-var-lib-cni-bin\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.909882 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909829    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/110466f9-e7b0-4a07-8a64-126c9ebaf9c1-serviceca\") pod \"node-ca-f8rvf\" (UID: \"110466f9-e7b0-4a07-8a64-126c9ebaf9c1\") " pod="openshift-image-registry/node-ca-f8rvf"
Jul 02 13:45:14.909882 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909861    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.910164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909901    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-cni-netd\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.910164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909926    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2539e33c-8342-4e24-879a-690d229ac2e9-env-overrides\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.910164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909952    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-multus-cni-dir\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.910164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-multus-socket-dir-parent\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.910164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.909994    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-lib-modules\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.910164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/247c1841-374e-4648-b211-aa3a08b28f06-cni-binary-copy\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.910164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910030    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-var-lib-kubelet\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.910164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910058    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a44edce6-3bb4-47ec-ac53-7724912c6277-konnectivity-ca\") pod \"konnectivity-agent-8sdgz\" (UID: \"a44edce6-3bb4-47ec-ac53-7724912c6277\") " pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:14.910164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/110466f9-e7b0-4a07-8a64-126c9ebaf9c1-host\") pod \"node-ca-f8rvf\" (UID: \"110466f9-e7b0-4a07-8a64-126c9ebaf9c1\") " pod="openshift-image-registry/node-ca-f8rvf"
Jul 02 13:45:14.910164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910096    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-run-netns\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.910164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910109    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-cni-bin\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910164    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/110466f9-e7b0-4a07-8a64-126c9ebaf9c1-host\") pod \"node-ca-f8rvf\" (UID: \"110466f9-e7b0-4a07-8a64-126c9ebaf9c1\") " pod="openshift-image-registry/node-ca-f8rvf"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910203    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-socket-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910229    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-system-cni-dir\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910250    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-run-netns\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910278    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/247c1841-374e-4648-b211-aa3a08b28f06-multus-daemon-config\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910307    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44gdt\" (UniqueName: \"kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt\") pod \"network-check-target-bt4j2\" (UID: \"afddd3e6-b476-40e8-95d8-2b2332236c91\") " pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-run\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910308    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/110466f9-e7b0-4a07-8a64-126c9ebaf9c1-serviceca\") pod \"node-ca-f8rvf\" (UID: \"110466f9-e7b0-4a07-8a64-126c9ebaf9c1\") " pod="openshift-image-registry/node-ca-f8rvf"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910346    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-tuned\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910427    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-os-release\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910457    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-run-multus-certs\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910482    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-node-log\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910509    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-device-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910538    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910538    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a44edce6-3bb4-47ec-ac53-7724912c6277-konnectivity-ca\") pod \"konnectivity-agent-8sdgz\" (UID: \"a44edce6-3bb4-47ec-ac53-7724912c6277\") " pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:14.910565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910572    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a44edce6-3bb4-47ec-ac53-7724912c6277-agent-certs\") pod \"konnectivity-agent-8sdgz\" (UID: \"a44edce6-3bb4-47ec-ac53-7724912c6277\") " pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910616    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/41ed71d6-1988-4c22-b3af-36711ebeef44-host-slash\") pod \"iptables-alerter-qzt8s\" (UID: \"41ed71d6-1988-4c22-b3af-36711ebeef44\") " pod="openshift-network-operator/iptables-alerter-qzt8s"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910648    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd9pw\" (UniqueName: \"kubernetes.io/projected/2539e33c-8342-4e24-879a-690d229ac2e9-kube-api-access-rd9pw\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910674    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw4zg\" (UniqueName: \"kubernetes.io/projected/747984cc-ab52-45dc-99ef-53c93e5bd101-kube-api-access-xw4zg\") pod \"node-resolver-zqz2f\" (UID: \"747984cc-ab52-45dc-99ef-53c93e5bd101\") " pod="openshift-dns/node-resolver-zqz2f"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.910684    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910702    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-system-cni-dir\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910729    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-var-lib-cni-multus\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910728    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/41ed71d6-1988-4c22-b3af-36711ebeef44-host-slash\") pod \"iptables-alerter-qzt8s\" (UID: \"41ed71d6-1988-4c22-b3af-36711ebeef44\") " pod="openshift-network-operator/iptables-alerter-qzt8s"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.910762    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret podName:49ea922e-112c-480c-9a70-7992f0e29149 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:15.410740726 +0000 UTC m=+1.943497370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret") pod "global-pull-secret-syncer-jbfz7" (UID: "49ea922e-112c-480c-9a70-7992f0e29149") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-hostroot\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910809    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-log-socket\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910832    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49ea922e-112c-480c-9a70-7992f0e29149-kubelet-config\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910886    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/41ed71d6-1988-4c22-b3af-36711ebeef44-iptables-alerter-script\") pod \"iptables-alerter-qzt8s\" (UID: \"41ed71d6-1988-4c22-b3af-36711ebeef44\") " pod="openshift-network-operator/iptables-alerter-qzt8s"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910902    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49ea922e-112c-480c-9a70-7992f0e29149-kubelet-config\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910901    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910933    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/747984cc-ab52-45dc-99ef-53c93e5bd101-hosts-file\") pod \"node-resolver-zqz2f\" (UID: \"747984cc-ab52-45dc-99ef-53c93e5bd101\") " pod="openshift-dns/node-resolver-zqz2f"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910972    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:14.911261 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.910996    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lrn5\" (UniqueName: \"kubernetes.io/projected/247c1841-374e-4648-b211-aa3a08b28f06-kube-api-access-7lrn5\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911012    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-run-ovn\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911035    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2539e33c-8342-4e24-879a-690d229ac2e9-ovn-node-metrics-cert\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911058    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-kubernetes\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911073    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747984cc-ab52-45dc-99ef-53c93e5bd101-tmp-dir\") pod \"node-resolver-zqz2f\" (UID: \"747984cc-ab52-45dc-99ef-53c93e5bd101\") " pod="openshift-dns/node-resolver-zqz2f"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911090    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f24875be-dc41-4b03-84e6-e00f7c5d9df0-cni-binary-copy\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911106    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-systemd-units\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911144    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2539e33c-8342-4e24-879a-690d229ac2e9-ovnkube-script-lib\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911172    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v5fp\" (UniqueName: \"kubernetes.io/projected/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-kube-api-access-5v5fp\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911236    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2539e33c-8342-4e24-879a-690d229ac2e9-ovnkube-config\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911257    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-sys-fs\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911277    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-sys\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911294    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/41ed71d6-1988-4c22-b3af-36711ebeef44-iptables-alerter-script\") pod \"iptables-alerter-qzt8s\" (UID: \"41ed71d6-1988-4c22-b3af-36711ebeef44\") " pod="openshift-network-operator/iptables-alerter-qzt8s"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f24875be-dc41-4b03-84e6-e00f7c5d9df0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.911321    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911333    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft5cv\" (UniqueName: \"kubernetes.io/projected/b0871170-b35a-4a3a-a915-dca9f7ce190b-kube-api-access-ft5cv\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:14.911836 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-cnibin\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.911376    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs podName:d01be1dd-2bb1-409d-aaaa-6094d6e05f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:15.411360957 +0000 UTC m=+1.944117591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs") pod "network-metrics-daemon-whrk8" (UID: "d01be1dd-2bb1-409d-aaaa-6094d6e05f26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911393    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49ea922e-112c-480c-9a70-7992f0e29149-dbus\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911418    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-etc-selinux\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911439    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-modprobe-d\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911459    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-sysctl-conf\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911477    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-var-lib-kubelet\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911496    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-etc-kubernetes\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911517    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-etc-openvswitch\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911535    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-run-openvswitch\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911553    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-registration-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911567    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49ea922e-112c-480c-9a70-7992f0e29149-dbus\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911572    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-sysctl-d\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911596    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-systemd\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911614    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-multus-conf-dir\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911637    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-run-systemd\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.912347 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.911660    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:14.914306 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.914285    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a44edce6-3bb4-47ec-ac53-7724912c6277-agent-certs\") pod \"konnectivity-agent-8sdgz\" (UID: \"a44edce6-3bb4-47ec-ac53-7724912c6277\") " pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:14.927832 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.927810    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46ldx\" (UniqueName: \"kubernetes.io/projected/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-kube-api-access-46ldx\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:14.928155 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.928138    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:45:14.928237 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.928162    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:45:14.928237 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.928176    2567 projected.go:196] Error preparing data for projected volume kube-api-access-44gdt for pod openshift-network-diagnostics/network-check-target-bt4j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:14.928329 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:14.928238    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt podName:afddd3e6-b476-40e8-95d8-2b2332236c91 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:15.428220658 +0000 UTC m=+1.960977291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-44gdt" (UniqueName: "kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt") pod "network-check-target-bt4j2" (UID: "afddd3e6-b476-40e8-95d8-2b2332236c91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:14.928329 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.928236    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvdkf\" (UniqueName: \"kubernetes.io/projected/41ed71d6-1988-4c22-b3af-36711ebeef44-kube-api-access-tvdkf\") pod \"iptables-alerter-qzt8s\" (UID: \"41ed71d6-1988-4c22-b3af-36711ebeef44\") " pod="openshift-network-operator/iptables-alerter-qzt8s"
Jul 02 13:45:14.929856 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:14.929828    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tlcc\" (UniqueName: \"kubernetes.io/projected/110466f9-e7b0-4a07-8a64-126c9ebaf9c1-kube-api-access-6tlcc\") pod \"node-ca-f8rvf\" (UID: \"110466f9-e7b0-4a07-8a64-126c9ebaf9c1\") " pod="openshift-image-registry/node-ca-f8rvf"
Jul 02 13:45:15.012448 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012412    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-lib-modules\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.012448 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012447    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/247c1841-374e-4648-b211-aa3a08b28f06-cni-binary-copy\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012467    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-var-lib-kubelet\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012492    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-run-netns\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012515    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-cni-bin\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012536    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-socket-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012544    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-var-lib-kubelet\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012556    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-system-cni-dir\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012566    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-run-netns\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012584    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-run-netns\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012595    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-cni-bin\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012606    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/247c1841-374e-4648-b211-aa3a08b28f06-multus-daemon-config\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012611    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-system-cni-dir\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012606    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-lib-modules\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012631    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-run-netns\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012681    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-run\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012684    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-socket-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012725    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-tuned\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012732    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-run\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.013285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012742    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-os-release\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012760    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-run-multus-certs\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012778    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-node-log\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-device-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012828    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd9pw\" (UniqueName: \"kubernetes.io/projected/2539e33c-8342-4e24-879a-690d229ac2e9-kube-api-access-rd9pw\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012839    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-os-release\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012840    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-run-multus-certs\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012873    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4zg\" (UniqueName: \"kubernetes.io/projected/747984cc-ab52-45dc-99ef-53c93e5bd101-kube-api-access-xw4zg\") pod \"node-resolver-zqz2f\" (UID: \"747984cc-ab52-45dc-99ef-53c93e5bd101\") " pod="openshift-dns/node-resolver-zqz2f"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012845    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-node-log\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012901    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-system-cni-dir\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012906    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-device-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012936    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-var-lib-cni-multus\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012949    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-system-cni-dir\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012960    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-hostroot\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-log-socket\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.012989    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-var-lib-cni-multus\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013011    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/747984cc-ab52-45dc-99ef-53c93e5bd101-hosts-file\") pod \"node-resolver-zqz2f\" (UID: \"747984cc-ab52-45dc-99ef-53c93e5bd101\") " pod="openshift-dns/node-resolver-zqz2f"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013024    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-hostroot\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.014203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013035    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013059    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-log-socket\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013066    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/747984cc-ab52-45dc-99ef-53c93e5bd101-hosts-file\") pod \"node-resolver-zqz2f\" (UID: \"747984cc-ab52-45dc-99ef-53c93e5bd101\") " pod="openshift-dns/node-resolver-zqz2f"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013068    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lrn5\" (UniqueName: \"kubernetes.io/projected/247c1841-374e-4648-b211-aa3a08b28f06-kube-api-access-7lrn5\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013078    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/247c1841-374e-4648-b211-aa3a08b28f06-cni-binary-copy\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-run-ovn\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013153    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-run-ovn\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013115    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/247c1841-374e-4648-b211-aa3a08b28f06-multus-daemon-config\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013148    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2539e33c-8342-4e24-879a-690d229ac2e9-ovn-node-metrics-cert\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013201    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013209    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-kubernetes\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747984cc-ab52-45dc-99ef-53c93e5bd101-tmp-dir\") pod \"node-resolver-zqz2f\" (UID: \"747984cc-ab52-45dc-99ef-53c93e5bd101\") " pod="openshift-dns/node-resolver-zqz2f"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013248    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-kubernetes\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f24875be-dc41-4b03-84e6-e00f7c5d9df0-cni-binary-copy\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-systemd-units\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013315    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2539e33c-8342-4e24-879a-690d229ac2e9-ovnkube-script-lib\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013341    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v5fp\" (UniqueName: \"kubernetes.io/projected/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-kube-api-access-5v5fp\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013378    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-systemd-units\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.015099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013382    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2539e33c-8342-4e24-879a-690d229ac2e9-ovnkube-config\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013430    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-sys-fs\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013454    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-sys\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013487    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f24875be-dc41-4b03-84e6-e00f7c5d9df0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013515    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft5cv\" (UniqueName: \"kubernetes.io/projected/b0871170-b35a-4a3a-a915-dca9f7ce190b-kube-api-access-ft5cv\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013541    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-cnibin\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013567    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-etc-selinux\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013590    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-modprobe-d\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013644    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-sysctl-conf\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013666    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-var-lib-kubelet\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013690    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-etc-kubernetes\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013515    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747984cc-ab52-45dc-99ef-53c93e5bd101-tmp-dir\") pod \"node-resolver-zqz2f\" (UID: \"747984cc-ab52-45dc-99ef-53c93e5bd101\") " pod="openshift-dns/node-resolver-zqz2f"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013713    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-etc-openvswitch\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013740    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-run-openvswitch\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013763    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-registration-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013787    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-sysctl-d\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013805    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-sys-fs\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.016039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-systemd\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013837    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f24875be-dc41-4b03-84e6-e00f7c5d9df0-cni-binary-copy\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013866    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-multus-conf-dir\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013874    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-sys\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013892    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-run-systemd\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013909    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2539e33c-8342-4e24-879a-690d229ac2e9-ovnkube-config\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013918    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013925    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2539e33c-8342-4e24-879a-690d229ac2e9-ovnkube-script-lib\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013927    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-etc-kubernetes\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013942    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013974    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-tmp\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.013988    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-var-lib-kubelet\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014001    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014041    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-etc-selinux\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014046    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-cnibin\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014045    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-run-openvswitch\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014089    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0871170-b35a-4a3a-a915-dca9f7ce190b-registration-dir\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.016656 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014128    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-sysctl-conf\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014129    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-multus-conf-dir\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014157    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-run-systemd\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014138    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014002    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-cnibin\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014198    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-run-k8s-cni-cncf-io\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014223    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-kubelet\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014239    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-modprobe-d\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014248    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-slash\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014274    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-os-release\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014289    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-run-k8s-cni-cncf-io\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014301    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtgvs\" (UniqueName: \"kubernetes.io/projected/f24875be-dc41-4b03-84e6-e00f7c5d9df0-kube-api-access-mtgvs\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014092    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-systemd\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014320    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-etc-openvswitch\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014333    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-var-lib-openvswitch\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014344    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-sysctl-d\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014358    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-sysconfig\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014382    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-host\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.017145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014357    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-slash\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014383    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-cnibin\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014302    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f24875be-dc41-4b03-84e6-e00f7c5d9df0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014405    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f24875be-dc41-4b03-84e6-e00f7c5d9df0-os-release\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014418    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-var-lib-openvswitch\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014358    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-kubelet\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014446    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f24875be-dc41-4b03-84e6-e00f7c5d9df0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014467    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-host\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014469    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-sysconfig\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014474    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-var-lib-cni-bin\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014505    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-host-var-lib-cni-bin\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014513    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-cni-netd\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014562    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2539e33c-8342-4e24-879a-690d229ac2e9-env-overrides\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014565    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-run-ovn-kubernetes\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014601    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2539e33c-8342-4e24-879a-690d229ac2e9-host-cni-netd\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014625    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-multus-cni-dir\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.017664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014663    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-multus-socket-dir-parent\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.018143 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014739    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-multus-socket-dir-parent\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.018143 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014762    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/247c1841-374e-4648-b211-aa3a08b28f06-multus-cni-dir\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.018143 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014903    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f24875be-dc41-4b03-84e6-e00f7c5d9df0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.018143 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.014924    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2539e33c-8342-4e24-879a-690d229ac2e9-env-overrides\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.018143 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.015389    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-etc-tuned\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.018143 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.015443    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2539e33c-8342-4e24-879a-690d229ac2e9-ovn-node-metrics-cert\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.018143 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.016461    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-tmp\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.033771 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.033748    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd9pw\" (UniqueName: \"kubernetes.io/projected/2539e33c-8342-4e24-879a-690d229ac2e9-kube-api-access-rd9pw\") pod \"ovnkube-node-xtms2\" (UID: \"2539e33c-8342-4e24-879a-690d229ac2e9\") " pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.033933 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.033748    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw4zg\" (UniqueName: \"kubernetes.io/projected/747984cc-ab52-45dc-99ef-53c93e5bd101-kube-api-access-xw4zg\") pod \"node-resolver-zqz2f\" (UID: \"747984cc-ab52-45dc-99ef-53c93e5bd101\") " pod="openshift-dns/node-resolver-zqz2f"
Jul 02 13:45:15.041153 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.041128    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lrn5\" (UniqueName: \"kubernetes.io/projected/247c1841-374e-4648-b211-aa3a08b28f06-kube-api-access-7lrn5\") pod \"multus-md66t\" (UID: \"247c1841-374e-4648-b211-aa3a08b28f06\") " pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.042885 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.042843    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtgvs\" (UniqueName: \"kubernetes.io/projected/f24875be-dc41-4b03-84e6-e00f7c5d9df0-kube-api-access-mtgvs\") pod \"multus-additional-cni-plugins-n88db\" (UID: \"f24875be-dc41-4b03-84e6-e00f7c5d9df0\") " pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.043417 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.043389    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft5cv\" (UniqueName: \"kubernetes.io/projected/b0871170-b35a-4a3a-a915-dca9f7ce190b-kube-api-access-ft5cv\") pod \"aws-ebs-csi-driver-node-r2v2f\" (UID: \"b0871170-b35a-4a3a-a915-dca9f7ce190b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.043599 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.043583    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v5fp\" (UniqueName: \"kubernetes.io/projected/2eab9828-8f6d-4792-a8ce-c1e5a354f6a5-kube-api-access-5v5fp\") pod \"tuned-jrhsd\" (UID: \"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5\") " pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.094225 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.094147    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:45:15.109755 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.109702    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 13:40:15 +0000 UTC" deadline="2027-12-03 10:07:15.477579699 +0000 UTC"
Jul 02 13:45:15.109755 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.109747    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12452h22m0.367835065s"
Jul 02 13:45:15.133489 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.133445    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:15.139585 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.139566    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f8rvf"
Jul 02 13:45:15.142429 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:15.142404    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda44edce6_3bb4_47ec_ac53_7724912c6277.slice/crio-ef1982d5019af4307a0de27014608b269b280b3ea246999d0a9af713d5a13a6e WatchSource:0}: Error finding container ef1982d5019af4307a0de27014608b269b280b3ea246999d0a9af713d5a13a6e: Status 404 returned error can't find the container with id ef1982d5019af4307a0de27014608b269b280b3ea246999d0a9af713d5a13a6e
Jul 02 13:45:15.147237 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:15.147215    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod110466f9_e7b0_4a07_8a64_126c9ebaf9c1.slice/crio-ba636c0b94a4937ccce469c895ba6a1c9415e075229000309e40e03cd55d53cc WatchSource:0}: Error finding container ba636c0b94a4937ccce469c895ba6a1c9415e075229000309e40e03cd55d53cc: Status 404 returned error can't find the container with id ba636c0b94a4937ccce469c895ba6a1c9415e075229000309e40e03cd55d53cc
Jul 02 13:45:15.147301 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.147267    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qzt8s"
Jul 02 13:45:15.151741 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.151725    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:15.155176 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:15.155150    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ed71d6_1988_4c22_b3af_36711ebeef44.slice/crio-e99ebcaa230443202c6aab860eba22ee16db51bc4ae8e05c544c8a7b7e638043 WatchSource:0}: Error finding container e99ebcaa230443202c6aab860eba22ee16db51bc4ae8e05c544c8a7b7e638043: Status 404 returned error can't find the container with id e99ebcaa230443202c6aab860eba22ee16db51bc4ae8e05c544c8a7b7e638043
Jul 02 13:45:15.157054 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.157033    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f"
Jul 02 13:45:15.159935 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:15.159914    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2539e33c_8342_4e24_879a_690d229ac2e9.slice/crio-51ac98c38cd32c1f3f4230c0b744964035ceaceea3375bf1384ed1dbbbd43588 WatchSource:0}: Error finding container 51ac98c38cd32c1f3f4230c0b744964035ceaceea3375bf1384ed1dbbbd43588: Status 404 returned error can't find the container with id 51ac98c38cd32c1f3f4230c0b744964035ceaceea3375bf1384ed1dbbbd43588
Jul 02 13:45:15.161793 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.161775    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jrhsd"
Jul 02 13:45:15.164587 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:15.164565    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0871170_b35a_4a3a_a915_dca9f7ce190b.slice/crio-3abf2be95a71f391f0e2f27aa36c684de5fda68d7d31e4606359dfbd9fb655dd WatchSource:0}: Error finding container 3abf2be95a71f391f0e2f27aa36c684de5fda68d7d31e4606359dfbd9fb655dd: Status 404 returned error can't find the container with id 3abf2be95a71f391f0e2f27aa36c684de5fda68d7d31e4606359dfbd9fb655dd
Jul 02 13:45:15.167102 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.167032    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zqz2f"
Jul 02 13:45:15.172375 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:15.172352    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eab9828_8f6d_4792_a8ce_c1e5a354f6a5.slice/crio-848fbdc39430f6761aca744ee3495db6df4663806448e03be60ecc93e48c6376 WatchSource:0}: Error finding container 848fbdc39430f6761aca744ee3495db6df4663806448e03be60ecc93e48c6376: Status 404 returned error can't find the container with id 848fbdc39430f6761aca744ee3495db6df4663806448e03be60ecc93e48c6376
Jul 02 13:45:15.172452 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.172414    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n88db"
Jul 02 13:45:15.175324 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:15.175293    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod747984cc_ab52_45dc_99ef_53c93e5bd101.slice/crio-da565c67670bf956a448f18ea1f70514d8fededf1c0185bd20cd438cd2abb91b WatchSource:0}: Error finding container da565c67670bf956a448f18ea1f70514d8fededf1c0185bd20cd438cd2abb91b: Status 404 returned error can't find the container with id da565c67670bf956a448f18ea1f70514d8fededf1c0185bd20cd438cd2abb91b
Jul 02 13:45:15.178813 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.178794    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-md66t"
Jul 02 13:45:15.180755 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:15.180734    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf24875be_dc41_4b03_84e6_e00f7c5d9df0.slice/crio-3c28354ff9685e706372390903b556752a84aceb76d747f3351d4f8164e843c5 WatchSource:0}: Error finding container 3c28354ff9685e706372390903b556752a84aceb76d747f3351d4f8164e843c5: Status 404 returned error can't find the container with id 3c28354ff9685e706372390903b556752a84aceb76d747f3351d4f8164e843c5
Jul 02 13:45:15.186195 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:15.186161    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod247c1841_374e_4648_b211_aa3a08b28f06.slice/crio-3f208b67ab8153db5be9a98d3d7da2b5333e40d61cd542e9a149e961e6f8b09c WatchSource:0}: Error finding container 3f208b67ab8153db5be9a98d3d7da2b5333e40d61cd542e9a149e961e6f8b09c: Status 404 returned error can't find the container with id 3f208b67ab8153db5be9a98d3d7da2b5333e40d61cd542e9a149e961e6f8b09c
Jul 02 13:45:15.247412 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.247379    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:45:15.417897 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.417789    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:15.417897 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:15.417865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:15.418116 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:15.417980    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:15.418116 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:15.417990    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:15.418116 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:15.418039    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs podName:d01be1dd-2bb1-409d-aaaa-6094d6e05f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:16.418022698 +0000 UTC m=+2.950779336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs") pod "network-metrics-daemon-whrk8" (UID: "d01be1dd-2bb1-409d-aaaa-6094d6e05f26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:15.418116 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:15.418053    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret podName:49ea922e-112c-480c-9a70-7992f0e29149 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:16.418046723 +0000 UTC m=+2.950803341 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret") pod "global-pull-secret-syncer-jbfz7" (UID: "49ea922e-112c-480c-9a70-7992f0e29149") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:16.546598 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.546343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44gdt\" (UniqueName: \"kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt\") pod \"network-check-target-bt4j2\" (UID: \"afddd3e6-b476-40e8-95d8-2b2332236c91\") " pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:16.546598 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:16.546567    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:45:16.546598 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:16.546586    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:45:16.546598 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:16.546598    2567 projected.go:196] Error preparing data for projected volume kube-api-access-44gdt for pod openshift-network-diagnostics/network-check-target-bt4j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:16.547023 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:16.546663    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt podName:afddd3e6-b476-40e8-95d8-2b2332236c91 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:17.546643453 +0000 UTC m=+3.051767955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-44gdt" (UniqueName: "kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt") pod "network-check-target-bt4j2" (UID: "afddd3e6-b476-40e8-95d8-2b2332236c91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:16.912818 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.912730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jrhsd" event={"ID":"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5","Type":"ContainerStarted","Data":"848fbdc39430f6761aca744ee3495db6df4663806448e03be60ecc93e48c6376"}
Jul 02 13:45:16.914998 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.914957    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 13:40:15 +0000 UTC" deadline="2028-04-13 00:04:09.932136668 +0000 UTC"
Jul 02 13:45:16.914998 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.914998    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15610h18m53.017143353s"
Jul 02 13:45:16.920619 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.920592    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2" event={"ID":"2539e33c-8342-4e24-879a-690d229ac2e9","Type":"ContainerStarted","Data":"51ac98c38cd32c1f3f4230c0b744964035ceaceea3375bf1384ed1dbbbd43588"}
Jul 02 13:45:16.932696 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.932601    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qzt8s" event={"ID":"41ed71d6-1988-4c22-b3af-36711ebeef44","Type":"ContainerStarted","Data":"e99ebcaa230443202c6aab860eba22ee16db51bc4ae8e05c544c8a7b7e638043"}
Jul 02 13:45:16.942402 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.942367    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f8rvf" event={"ID":"110466f9-e7b0-4a07-8a64-126c9ebaf9c1","Type":"ContainerStarted","Data":"ba636c0b94a4937ccce469c895ba6a1c9415e075229000309e40e03cd55d53cc"}
Jul 02 13:45:16.947112 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.947074    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8sdgz" event={"ID":"a44edce6-3bb4-47ec-ac53-7724912c6277","Type":"ContainerStarted","Data":"ef1982d5019af4307a0de27014608b269b280b3ea246999d0a9af713d5a13a6e"}
Jul 02 13:45:16.956047 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.956000    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-md66t" event={"ID":"247c1841-374e-4648-b211-aa3a08b28f06","Type":"ContainerStarted","Data":"3f208b67ab8153db5be9a98d3d7da2b5333e40d61cd542e9a149e961e6f8b09c"}
Jul 02 13:45:16.963917 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.963880    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n88db" event={"ID":"f24875be-dc41-4b03-84e6-e00f7c5d9df0","Type":"ContainerStarted","Data":"3c28354ff9685e706372390903b556752a84aceb76d747f3351d4f8164e843c5"}
Jul 02 13:45:16.970829 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.970792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zqz2f" event={"ID":"747984cc-ab52-45dc-99ef-53c93e5bd101","Type":"ContainerStarted","Data":"da565c67670bf956a448f18ea1f70514d8fededf1c0185bd20cd438cd2abb91b"}
Jul 02 13:45:16.980010 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:16.979952    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f" event={"ID":"b0871170-b35a-4a3a-a915-dca9f7ce190b","Type":"ContainerStarted","Data":"3abf2be95a71f391f0e2f27aa36c684de5fda68d7d31e4606359dfbd9fb655dd"}
Jul 02 13:45:17.453993 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:17.453954    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:17.454185 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:17.454067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:17.454254 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:17.454190    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:17.454254 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:17.454250    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret podName:49ea922e-112c-480c-9a70-7992f0e29149 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:19.454232198 +0000 UTC m=+4.959356687 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret") pod "global-pull-secret-syncer-jbfz7" (UID: "49ea922e-112c-480c-9a70-7992f0e29149") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:17.454364 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:17.454316    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:17.454364 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:17.454348    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs podName:d01be1dd-2bb1-409d-aaaa-6094d6e05f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:19.45433635 +0000 UTC m=+4.959460836 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs") pod "network-metrics-daemon-whrk8" (UID: "d01be1dd-2bb1-409d-aaaa-6094d6e05f26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:17.554467 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:17.554415    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44gdt\" (UniqueName: \"kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt\") pod \"network-check-target-bt4j2\" (UID: \"afddd3e6-b476-40e8-95d8-2b2332236c91\") " pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:17.555135 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:17.554676    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:45:17.555135 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:17.554699    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:45:17.555135 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:17.554712    2567 projected.go:196] Error preparing data for projected volume kube-api-access-44gdt for pod openshift-network-diagnostics/network-check-target-bt4j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:17.555135 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:17.554773    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt podName:afddd3e6-b476-40e8-95d8-2b2332236c91 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:19.554754068 +0000 UTC m=+5.059878572 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-44gdt" (UniqueName: "kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt") pod "network-check-target-bt4j2" (UID: "afddd3e6-b476-40e8-95d8-2b2332236c91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:17.898787 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:17.898704    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:17.898974 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:17.898843    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:17.899294 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:17.899258    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:17.899399 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:17.899367    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:17.899453 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:17.899435    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:17.899554 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:17.899535    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:17.915272 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:17.915226    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 13:40:15 +0000 UTC" deadline="2028-03-03 08:15:59.658484194 +0000 UTC"
Jul 02 13:45:17.915760 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:17.915288    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14634h30m41.743201492s"
Jul 02 13:45:18.738374 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:18.738295    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:45:19.471284 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:19.470641    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:19.471284 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:19.470727    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:19.471284 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:19.470862    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:19.471284 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:19.470890    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:19.471284 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:19.470930    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret podName:49ea922e-112c-480c-9a70-7992f0e29149 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:23.47091187 +0000 UTC m=+8.976036354 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret") pod "global-pull-secret-syncer-jbfz7" (UID: "49ea922e-112c-480c-9a70-7992f0e29149") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:19.471284 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:19.470956    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs podName:d01be1dd-2bb1-409d-aaaa-6094d6e05f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:23.470938346 +0000 UTC m=+8.976062841 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs") pod "network-metrics-daemon-whrk8" (UID: "d01be1dd-2bb1-409d-aaaa-6094d6e05f26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:19.571809 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:19.571748    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44gdt\" (UniqueName: \"kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt\") pod \"network-check-target-bt4j2\" (UID: \"afddd3e6-b476-40e8-95d8-2b2332236c91\") " pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:19.572071 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:19.571948    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:45:19.572071 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:19.571969    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:45:19.572071 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:19.571982    2567 projected.go:196] Error preparing data for projected volume kube-api-access-44gdt for pod openshift-network-diagnostics/network-check-target-bt4j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:19.572071 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:19.572042    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt podName:afddd3e6-b476-40e8-95d8-2b2332236c91 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:23.572024129 +0000 UTC m=+9.077148628 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-44gdt" (UniqueName: "kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt") pod "network-check-target-bt4j2" (UID: "afddd3e6-b476-40e8-95d8-2b2332236c91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:19.899037 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:19.898162    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:19.899037 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:19.898300    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:19.899037 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:19.898704    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:19.899037 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:19.898798    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:19.899037 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:19.898859    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:19.899037 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:19.898941    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:21.898337 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:21.898297    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:21.898815 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:21.898439    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:21.898885 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:21.898840    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:21.899088 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:21.898944    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:21.899088 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:21.898990    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:21.899088 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:21.899052    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:23.503912 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:23.503759    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:23.503912 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:23.503833    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:23.504433 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:23.503961    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:23.504433 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:23.503988    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:23.504433 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:23.504037    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs podName:d01be1dd-2bb1-409d-aaaa-6094d6e05f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:31.504015614 +0000 UTC m=+17.009140109 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs") pod "network-metrics-daemon-whrk8" (UID: "d01be1dd-2bb1-409d-aaaa-6094d6e05f26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:23.504433 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:23.504056    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret podName:49ea922e-112c-480c-9a70-7992f0e29149 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:31.50404608 +0000 UTC m=+17.009170565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret") pod "global-pull-secret-syncer-jbfz7" (UID: "49ea922e-112c-480c-9a70-7992f0e29149") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:23.604320 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:23.604179    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44gdt\" (UniqueName: \"kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt\") pod \"network-check-target-bt4j2\" (UID: \"afddd3e6-b476-40e8-95d8-2b2332236c91\") " pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:23.604532 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:23.604408    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:45:23.604532 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:23.604435    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:45:23.604532 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:23.604449    2567 projected.go:196] Error preparing data for projected volume kube-api-access-44gdt for pod openshift-network-diagnostics/network-check-target-bt4j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:23.604532 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:23.604525    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt podName:afddd3e6-b476-40e8-95d8-2b2332236c91 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:31.604506009 +0000 UTC m=+17.109630495 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-44gdt" (UniqueName: "kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt") pod "network-check-target-bt4j2" (UID: "afddd3e6-b476-40e8-95d8-2b2332236c91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:23.899163 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:23.898349    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:23.899163 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:23.898492    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:23.899163 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:23.898570    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:23.899163 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:23.898655    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:23.899163 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:23.898708    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:23.899163 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:23.898775    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:25.897789 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:25.897745    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:25.897789 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:25.897788    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:25.898277 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:25.897753    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:25.898277 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:25.897896    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:25.898277 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:25.898020    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:25.898277 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:25.898110    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:27.898767 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:27.898573    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:27.899200 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:27.898573    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:27.899200 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:27.898854    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:27.899200 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:27.898580    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:27.899200 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:27.898982    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:27.899200 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:27.899032    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:29.898019 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:29.897980    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:29.898019 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:29.898005    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:29.898496 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:29.898015    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:29.898496 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:29.898116    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:29.898496 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:29.898172    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:29.898496 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:29.898263    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:31.568290 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:31.568195    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:31.568290 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:31.568261    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:31.568901 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:31.568372    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:31.568901 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:31.568471    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret podName:49ea922e-112c-480c-9a70-7992f0e29149 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:47.56844639 +0000 UTC m=+33.073570886 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret") pod "global-pull-secret-syncer-jbfz7" (UID: "49ea922e-112c-480c-9a70-7992f0e29149") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:31.568901 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:31.568632    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:31.568901 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:31.568707    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs podName:d01be1dd-2bb1-409d-aaaa-6094d6e05f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:47.568693845 +0000 UTC m=+33.073818346 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs") pod "network-metrics-daemon-whrk8" (UID: "d01be1dd-2bb1-409d-aaaa-6094d6e05f26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:31.668712 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:31.668666    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44gdt\" (UniqueName: \"kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt\") pod \"network-check-target-bt4j2\" (UID: \"afddd3e6-b476-40e8-95d8-2b2332236c91\") " pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:31.668900 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:31.668855    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:45:31.668900 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:31.668877    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:45:31.668900 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:31.668887    2567 projected.go:196] Error preparing data for projected volume kube-api-access-44gdt for pod openshift-network-diagnostics/network-check-target-bt4j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:31.669041 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:31.668947    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt podName:afddd3e6-b476-40e8-95d8-2b2332236c91 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:47.668929245 +0000 UTC m=+33.174053746 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-44gdt" (UniqueName: "kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt") pod "network-check-target-bt4j2" (UID: "afddd3e6-b476-40e8-95d8-2b2332236c91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:31.897887 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:31.897805    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:31.897887 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:31.897817    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:31.898083 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:31.897865    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:31.898083 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:31.897933    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:31.898083 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:31.898063    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:31.898193 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:31.898138    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:33.898522 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:33.898472    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:33.898522 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:33.898517    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:33.899073 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:33.898473    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:33.899073 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:33.898617    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:33.899073 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:33.898741    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:33.899073 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:33.898827    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:35.897986 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:35.897962    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:35.898268 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:35.897995    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:35.898268 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:35.898060    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:35.898268 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:35.898065    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:35.898268 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:35.898130    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:35.898268 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:35.898202    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:37.022788 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.022327    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal" event={"ID":"f18bf94e33493ecb15787b6bbd7e24d0","Type":"ContainerStarted","Data":"63d2e7aa53c62de66beee417f768aec65c412bbde628232389cc265f2dfde5c2"}
Jul 02 13:45:37.024227 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.024199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-md66t" event={"ID":"247c1841-374e-4648-b211-aa3a08b28f06","Type":"ContainerStarted","Data":"0a0005618b3b4b1fc0e04219aaadbc39151697090a2252249ff47ea0a7415551"}
Jul 02 13:45:37.025766 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.025745    2567 generic.go:356] "Generic (PLEG): container finished" podID="f24875be-dc41-4b03-84e6-e00f7c5d9df0" containerID="a7c7323920da4aedcfdd3703bfb0813d63cc9452779a3d7ed9a89ec8c4764049" exitCode=0
Jul 02 13:45:37.025881 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.025825    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n88db" event={"ID":"f24875be-dc41-4b03-84e6-e00f7c5d9df0","Type":"ContainerDied","Data":"a7c7323920da4aedcfdd3703bfb0813d63cc9452779a3d7ed9a89ec8c4764049"}
Jul 02 13:45:37.027535 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.027511    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zqz2f" event={"ID":"747984cc-ab52-45dc-99ef-53c93e5bd101","Type":"ContainerStarted","Data":"17ac9ee8fd51a97fdaea3a8c66264f1b3c78af72a27e58f0122e41b88acdd278"}
Jul 02 13:45:37.029160 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.029139    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f" event={"ID":"b0871170-b35a-4a3a-a915-dca9f7ce190b","Type":"ContainerStarted","Data":"7b8e326ce982da346d5d7b3c5af59c8fb491f0536f48f34ea8a9a194e2bf6b16"}
Jul 02 13:45:37.030735 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.030714    2567 generic.go:356] "Generic (PLEG): container finished" podID="fa3e9918aa0232d73fb5a4effb0efd9d" containerID="3c02378786d8f165c224c20e20e8cebe35aa4bfc33dd0a3300d1609ae0bb2fed" exitCode=0
Jul 02 13:45:37.030830 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.030791    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal" event={"ID":"fa3e9918aa0232d73fb5a4effb0efd9d","Type":"ContainerDied","Data":"3c02378786d8f165c224c20e20e8cebe35aa4bfc33dd0a3300d1609ae0bb2fed"}
Jul 02 13:45:37.032374 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.032338    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jrhsd" event={"ID":"2eab9828-8f6d-4792-a8ce-c1e5a354f6a5","Type":"ContainerStarted","Data":"bea749f60d96f78f85605d4f4460c83af89f5836e8efab74256710839ec4e77d"}
Jul 02 13:45:37.035373 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.035354    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 13:45:37.035775 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.035753    2567 generic.go:356] "Generic (PLEG): container finished" podID="2539e33c-8342-4e24-879a-690d229ac2e9" containerID="dac0548e1ae2ef713d81755510eaa6aec85fdf799f39823f93d11239cdfbf516" exitCode=1
Jul 02 13:45:37.035865 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.035817    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2" event={"ID":"2539e33c-8342-4e24-879a-690d229ac2e9","Type":"ContainerStarted","Data":"7505e6b0cc6081bf57a87859cba7d1cc50bdb21e0b10491bd0ddd7ba9a60fc4c"}
Jul 02 13:45:37.035904 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.035869    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2" event={"ID":"2539e33c-8342-4e24-879a-690d229ac2e9","Type":"ContainerStarted","Data":"6f69607e4dd882636437a9eca7cd365f47b49793f11bf4238a361b1d1810e550"}
Jul 02 13:45:37.035904 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.035899    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2" event={"ID":"2539e33c-8342-4e24-879a-690d229ac2e9","Type":"ContainerStarted","Data":"cf54779c145fcb99f2e8040ee9cc766b3abc00fc727cf377f3f398482b4fc17c"}
Jul 02 13:45:37.035988 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.035931    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2" event={"ID":"2539e33c-8342-4e24-879a-690d229ac2e9","Type":"ContainerStarted","Data":"65d37158e5cfa588b25ad9798e0b1df080a3c595d3562fbdf7936315cfe1d3f7"}
Jul 02 13:45:37.035988 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.035971    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2" event={"ID":"2539e33c-8342-4e24-879a-690d229ac2e9","Type":"ContainerDied","Data":"dac0548e1ae2ef713d81755510eaa6aec85fdf799f39823f93d11239cdfbf516"}
Jul 02 13:45:37.036080 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.036018    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2" event={"ID":"2539e33c-8342-4e24-879a-690d229ac2e9","Type":"ContainerStarted","Data":"471467aa2cc136107faf1639251aabe9e1d6250deabbe231f5e3f58014e26a16"}
Jul 02 13:45:37.037532 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.037507    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f8rvf" event={"ID":"110466f9-e7b0-4a07-8a64-126c9ebaf9c1","Type":"ContainerStarted","Data":"509d6f04342f7e73cf390249b47c7db7705e92a0033eef2e94e5d8bddad8676d"}
Jul 02 13:45:37.039013 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.038994    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8sdgz" event={"ID":"a44edce6-3bb4-47ec-ac53-7724912c6277","Type":"ContainerStarted","Data":"e8cc9b6bd86e90d7d7bb324921b246af2d8d7626bfa1132b50d800dec09ff2d9"}
Jul 02 13:45:37.063642 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.063588    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-226.ec2.internal" podStartSLOduration=22.063567993 podStartE2EDuration="22.063567993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:37.042208408 +0000 UTC m=+22.547332938" watchObservedRunningTime="2026-07-02 13:45:37.063567993 +0000 UTC m=+22.568692497"
Jul 02 13:45:37.064125 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.064089    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8sdgz" podStartSLOduration=2.455094334 podStartE2EDuration="22.064076104s" totalImagesPullingTime="19.60898177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:37.063963593 +0000 UTC m=+22.569088127" watchObservedRunningTime="2026-07-02 13:45:37.064076104 +0000 UTC m=+22.569200609"
Jul 02 13:45:37.083946 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.083806    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f8rvf" podStartSLOduration=6.523296999 podStartE2EDuration="22.083784645s" totalImagesPullingTime="15.560487646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:37.083713954 +0000 UTC m=+22.588838459" watchObservedRunningTime="2026-07-02 13:45:37.083784645 +0000 UTC m=+22.588909154"
Jul 02 13:45:37.167840 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.167789    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jrhsd" podStartSLOduration=2.550613876 podStartE2EDuration="22.167766749s" totalImagesPullingTime="19.617152873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:37.127231169 +0000 UTC m=+22.632355672" watchObservedRunningTime="2026-07-02 13:45:37.167766749 +0000 UTC m=+22.672891254"
Jul 02 13:45:37.247576 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.247511    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-md66t" podStartSLOduration=2.069876196 podStartE2EDuration="22.247470516s" totalImagesPullingTime="20.17759432s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:37.199966453 +0000 UTC m=+22.705090971" watchObservedRunningTime="2026-07-02 13:45:37.247470516 +0000 UTC m=+22.752595023"
Jul 02 13:45:37.321550 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.321509    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:37.529135 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.529056    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:37.775924 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.775893    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 02 13:45:37.888547 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.888437    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T13:45:37.775915809Z","UUID":"bb5694de-77df-422b-93e3-f147b335c1bd","Handler":null,"Name":"","Endpoint":""}
Jul 02 13:45:37.890275 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.890239    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 02 13:45:37.890420 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.890286    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 02 13:45:37.898494 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.898460    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:37.898641 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.898460    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:37.898641 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.898599    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:37.898641 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:37.898603    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:37.898772 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:37.898693    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:37.898822 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:37.898779    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:37.952120 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:37.952077    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:38.043354 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:38.043259    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qzt8s" event={"ID":"41ed71d6-1988-4c22-b3af-36711ebeef44","Type":"ContainerStarted","Data":"7b1f57b54e88c7bce34f5478c8fbf6f4cf1bb2f4a7f7cdd2b504870f5b72fc40"}
Jul 02 13:45:38.045612 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:38.045586    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f" event={"ID":"b0871170-b35a-4a3a-a915-dca9f7ce190b","Type":"ContainerStarted","Data":"e37ac0cee6a6dc4c1170d32e572ac7a74b0e6d3b8175c551fc8c03e063535940"}
Jul 02 13:45:38.093429 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:38.093368    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zqz2f" podStartSLOduration=3.517666975 podStartE2EDuration="23.093348362s" totalImagesPullingTime="19.575681387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:37.253308498 +0000 UTC m=+22.758432999" watchObservedRunningTime="2026-07-02 13:45:38.093348362 +0000 UTC m=+23.598472867"
Jul 02 13:45:38.124432 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:38.124375    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qzt8s" podStartSLOduration=3.490245101 podStartE2EDuration="23.124357193s" totalImagesPullingTime="19.634112092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:38.092979172 +0000 UTC m=+23.598103698" watchObservedRunningTime="2026-07-02 13:45:38.124357193 +0000 UTC m=+23.629481697"
Jul 02 13:45:38.314523 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:38.314429    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:38.495962 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:38.495929    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:38.578926 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:38.578850    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:39.049915 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:39.049882    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal" event={"ID":"fa3e9918aa0232d73fb5a4effb0efd9d","Type":"ContainerStarted","Data":"4d46f6a50eb678a6c0dc131cec88b4d14222ea2671a1260b7e8fdc03b232151d"}
Jul 02 13:45:39.053423 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:39.053402    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 13:45:39.053865 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:39.053829    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2" event={"ID":"2539e33c-8342-4e24-879a-690d229ac2e9","Type":"ContainerStarted","Data":"e0fddd50d9f26811268ad30b32dfc03a83228c02e565e9d0677c0d03876b0775"}
Jul 02 13:45:39.072450 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:39.072400    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-226.ec2.internal" podStartSLOduration=24.072384303 podStartE2EDuration="24.072384303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:39.071956921 +0000 UTC m=+24.577081427" watchObservedRunningTime="2026-07-02 13:45:39.072384303 +0000 UTC m=+24.577508808"
Jul 02 13:45:39.740045 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:39.740010    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:39.740635 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:39.740613    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:39.898702 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:39.898667    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:39.898702 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:39.898694    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:39.898953 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:39.898679    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:39.898953 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:39.898804    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:39.898953 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:39.898901    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:39.899091 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:39.898979    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:40.058725 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:40.058627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f" event={"ID":"b0871170-b35a-4a3a-a915-dca9f7ce190b","Type":"ContainerStarted","Data":"5c51d6cff52c06dedbfea7aea67e5e18053a83a4a72863b22eda1563b513b4a6"}
Jul 02 13:45:40.059171 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:40.058872    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:40.059545 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:40.059524    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8sdgz"
Jul 02 13:45:40.083279 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:40.083223    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2v2f" podStartSLOduration=3.286389412 podStartE2EDuration="25.083207238s" totalImagesPullingTime="21.796817826s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:40.082918803 +0000 UTC m=+25.588043306" watchObservedRunningTime="2026-07-02 13:45:40.083207238 +0000 UTC m=+25.588331742"
Jul 02 13:45:40.158222 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:40.158160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:41.898530 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:41.898256    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:41.898530 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:41.898283    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:41.899334 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:41.898555    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:41.899334 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:41.898256    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:41.899334 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:41.898602    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:41.899334 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:41.898672    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:42.064241 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:42.064212    2567 generic.go:356] "Generic (PLEG): container finished" podID="f24875be-dc41-4b03-84e6-e00f7c5d9df0" containerID="9ead023a2d0f2f0795a0279d7bbe813970c253ef6beb865bc4cd72480615277f" exitCode=0
Jul 02 13:45:42.064409 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:42.064266    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n88db" event={"ID":"f24875be-dc41-4b03-84e6-e00f7c5d9df0","Type":"ContainerDied","Data":"9ead023a2d0f2f0795a0279d7bbe813970c253ef6beb865bc4cd72480615277f"}
Jul 02 13:45:42.068004 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:42.067798    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 13:45:42.068352 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:42.068324    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2" event={"ID":"2539e33c-8342-4e24-879a-690d229ac2e9","Type":"ContainerStarted","Data":"4e9c9e3a51afbd048aa4a22f4baa33b87904ebbadb27a3af14f6330ca38be598"}
Jul 02 13:45:42.068602 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:42.068577    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:42.068873 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:42.068652    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:42.068873 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:42.068704    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:42.068873 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:42.068786    2567 scope.go:117] "RemoveContainer" containerID="dac0548e1ae2ef713d81755510eaa6aec85fdf799f39823f93d11239cdfbf516"
Jul 02 13:45:42.084678 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:42.084649    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:42.084812 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:42.084773    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:45:43.074938 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:43.074867    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 13:45:43.075438 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:43.075321    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2" event={"ID":"2539e33c-8342-4e24-879a-690d229ac2e9","Type":"ContainerStarted","Data":"5b19e6d1eba84811902c60745aa308ee9ff78173fe7503e2005fd04a1c0e1022"}
Jul 02 13:45:43.110204 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:43.110145    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2" podStartSLOduration=8.457288712 podStartE2EDuration="28.110129326s" totalImagesPullingTime="19.652840614s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:43.10952264 +0000 UTC m=+28.614647166" watchObservedRunningTime="2026-07-02 13:45:43.110129326 +0000 UTC m=+28.615253829"
Jul 02 13:45:43.219993 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:43.219956    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jbfz7"]
Jul 02 13:45:43.220175 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:43.220090    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:43.220241 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:43.220205    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:43.223642 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:43.223611    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bt4j2"]
Jul 02 13:45:43.223804 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:43.223740    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:43.223862 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:43.223840    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:43.224342 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:43.224320    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-whrk8"]
Jul 02 13:45:43.224460 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:43.224431    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:43.224569 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:43.224550    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:44.079037 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:44.078940    2567 generic.go:356] "Generic (PLEG): container finished" podID="f24875be-dc41-4b03-84e6-e00f7c5d9df0" containerID="9f6633394b44edb5dcffeca19d8a755bf686ad8cd3471ab71c5a857334df42b4" exitCode=0
Jul 02 13:45:44.079454 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:44.079030    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n88db" event={"ID":"f24875be-dc41-4b03-84e6-e00f7c5d9df0","Type":"ContainerDied","Data":"9f6633394b44edb5dcffeca19d8a755bf686ad8cd3471ab71c5a857334df42b4"}
Jul 02 13:45:44.899007 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:44.898975    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:44.899146 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:44.899078    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:44.899197 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:44.899161    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:44.899296 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:44.899275    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:44.899336 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:44.899324    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:44.899422 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:44.899407    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:46.085304 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:46.085117    2567 generic.go:356] "Generic (PLEG): container finished" podID="f24875be-dc41-4b03-84e6-e00f7c5d9df0" containerID="32c870d76a24d131fc85282d8cec93746aff136fd2608e73a4d9eb53f191d026" exitCode=0
Jul 02 13:45:46.085710 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:46.085204    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n88db" event={"ID":"f24875be-dc41-4b03-84e6-e00f7c5d9df0","Type":"ContainerDied","Data":"32c870d76a24d131fc85282d8cec93746aff136fd2608e73a4d9eb53f191d026"}
Jul 02 13:45:46.897890 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:46.897853    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:46.898060 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:46.897853    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:46.898060 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:46.897994    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:46.898166 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:46.898060    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:46.898166 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:46.897857    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:46.898166 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:46.898160    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:47.590187 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:47.590153    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:47.590651 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:47.590253    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:47.590651 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:47.590323    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:47.590651 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:47.590374    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:47.590651 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:47.590408    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs podName:d01be1dd-2bb1-409d-aaaa-6094d6e05f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:19.590387025 +0000 UTC m=+65.095511513 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs") pod "network-metrics-daemon-whrk8" (UID: "d01be1dd-2bb1-409d-aaaa-6094d6e05f26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:45:47.590651 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:47.590429    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret podName:49ea922e-112c-480c-9a70-7992f0e29149 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:19.590419602 +0000 UTC m=+65.095544087 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret") pod "global-pull-secret-syncer-jbfz7" (UID: "49ea922e-112c-480c-9a70-7992f0e29149") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:45:47.691087 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:47.691054    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44gdt\" (UniqueName: \"kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt\") pod \"network-check-target-bt4j2\" (UID: \"afddd3e6-b476-40e8-95d8-2b2332236c91\") " pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:47.691254 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:47.691189    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:45:47.691254 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:47.691207    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:45:47.691254 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:47.691217    2567 projected.go:196] Error preparing data for projected volume kube-api-access-44gdt for pod openshift-network-diagnostics/network-check-target-bt4j2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:47.691422 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:47.691267    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt podName:afddd3e6-b476-40e8-95d8-2b2332236c91 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:19.691253486 +0000 UTC m=+65.196377982 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-44gdt" (UniqueName: "kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt") pod "network-check-target-bt4j2" (UID: "afddd3e6-b476-40e8-95d8-2b2332236c91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:45:48.128384 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:48.128349    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:48.187688 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:48.187656    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:48.898243 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:48.898201    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:48.898243 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:48.898249    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:48.898932 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:48.898201    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:48.898932 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:48.898339    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-whrk8" podUID="d01be1dd-2bb1-409d-aaaa-6094d6e05f26"
Jul 02 13:45:48.898932 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:48.898400    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbfz7" podUID="49ea922e-112c-480c-9a70-7992f0e29149"
Jul 02 13:45:48.898932 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:48.898475    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bt4j2" podUID="afddd3e6-b476-40e8-95d8-2b2332236c91"
Jul 02 13:45:49.257924 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.257849    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-226.ec2.internal" event="NodeReady"
Jul 02 13:45:49.258057 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.257990    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 02 13:45:49.318314 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.318271    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kfvfm"]
Jul 02 13:45:49.320708 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.320687    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kfvfm"
Jul 02 13:45:49.323429 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.323403    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9h5lq"]
Jul 02 13:45:49.324237 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.324211    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 02 13:45:49.324432 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.324406    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mrnkk\""
Jul 02 13:45:49.324559 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.324446    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 02 13:45:49.325378 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.325360    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.325958 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.325929    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 02 13:45:49.328130 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.328110    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 02 13:45:49.328381 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.328364    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pnc7w\""
Jul 02 13:45:49.328647 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.328634    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 02 13:45:49.335626 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.335598    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kfvfm"]
Jul 02 13:45:49.338870 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.338654    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9h5lq"]
Jul 02 13:45:49.404520 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.404470    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.404695 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.404556    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dhsm\" (UniqueName: \"kubernetes.io/projected/99d73dda-2da1-400e-bc32-a373ba20f546-kube-api-access-4dhsm\") pod \"ingress-canary-kfvfm\" (UID: \"99d73dda-2da1-400e-bc32-a373ba20f546\") " pod="openshift-ingress-canary/ingress-canary-kfvfm"
Jul 02 13:45:49.404695 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.404599    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99d73dda-2da1-400e-bc32-a373ba20f546-cert\") pod \"ingress-canary-kfvfm\" (UID: \"99d73dda-2da1-400e-bc32-a373ba20f546\") " pod="openshift-ingress-canary/ingress-canary-kfvfm"
Jul 02 13:45:49.404695 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.404625    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2k9j\" (UniqueName: \"kubernetes.io/projected/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-kube-api-access-j2k9j\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.404695 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.404654    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-config-volume\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.404897 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.404712    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-tmp-dir\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.496408 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.496373    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:49.505270 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.505242    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-tmp-dir\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.505432 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.505278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.505432 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.505318    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dhsm\" (UniqueName: \"kubernetes.io/projected/99d73dda-2da1-400e-bc32-a373ba20f546-kube-api-access-4dhsm\") pod \"ingress-canary-kfvfm\" (UID: \"99d73dda-2da1-400e-bc32-a373ba20f546\") " pod="openshift-ingress-canary/ingress-canary-kfvfm"
Jul 02 13:45:49.505432 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.505357    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99d73dda-2da1-400e-bc32-a373ba20f546-cert\") pod \"ingress-canary-kfvfm\" (UID: \"99d73dda-2da1-400e-bc32-a373ba20f546\") " pod="openshift-ingress-canary/ingress-canary-kfvfm"
Jul 02 13:45:49.505432 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.505379    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2k9j\" (UniqueName: \"kubernetes.io/projected/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-kube-api-access-j2k9j\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.505432 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.505407    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-config-volume\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.505432 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:49.505421    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:45:49.505759 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:49.505519    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls podName:b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:50.005496817 +0000 UTC m=+35.510621319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls") pod "dns-default-9h5lq" (UID: "b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6") : secret "dns-default-metrics-tls" not found
Jul 02 13:45:49.505759 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.505675    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-tmp-dir\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.506110 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.506089    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-config-volume\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.510392 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.510341    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99d73dda-2da1-400e-bc32-a373ba20f546-cert\") pod \"ingress-canary-kfvfm\" (UID: \"99d73dda-2da1-400e-bc32-a373ba20f546\") " pod="openshift-ingress-canary/ingress-canary-kfvfm"
Jul 02 13:45:49.537574 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.537537    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dhsm\" (UniqueName: \"kubernetes.io/projected/99d73dda-2da1-400e-bc32-a373ba20f546-kube-api-access-4dhsm\") pod \"ingress-canary-kfvfm\" (UID: \"99d73dda-2da1-400e-bc32-a373ba20f546\") " pod="openshift-ingress-canary/ingress-canary-kfvfm"
Jul 02 13:45:49.542822 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.542792    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2k9j\" (UniqueName: \"kubernetes.io/projected/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-kube-api-access-j2k9j\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:49.627452 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.627418    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:49.633099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:49.633070    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kfvfm"
Jul 02 13:45:50.008977 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:50.008952    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:50.009657 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:50.009100    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:45:50.009657 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:50.009173    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls podName:b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:51.00914752 +0000 UTC m=+36.514272002 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls") pod "dns-default-9h5lq" (UID: "b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6") : secret "dns-default-metrics-tls" not found
Jul 02 13:45:50.124305 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:50.124112    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kfvfm"]
Jul 02 13:45:50.127823 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:45:50.127789    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d73dda_2da1_400e_bc32_a373ba20f546.slice/crio-f5c48c618809b16fae8840cddedd1f04995e51a9e638b56d4457c6633154f797 WatchSource:0}: Error finding container f5c48c618809b16fae8840cddedd1f04995e51a9e638b56d4457c6633154f797: Status 404 returned error can't find the container with id f5c48c618809b16fae8840cddedd1f04995e51a9e638b56d4457c6633154f797
Jul 02 13:45:50.898471 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:50.898433    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:45:50.898471 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:50.898466    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:45:50.898728 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:50.898433    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:45:50.905716 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:50.905692    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 13:45:50.905716 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:50.905704    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 13:45:50.905904 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:50.905704    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 13:45:50.906284 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:50.906265    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 13:45:50.906668 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:50.906650    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4vzgw\""
Jul 02 13:45:50.906761 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:50.906690    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7vs28\""
Jul 02 13:45:51.015640 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:51.015604    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:51.016107 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:51.015741    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:45:51.016107 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:51.015815    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls podName:b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:53.01579621 +0000 UTC m=+38.520920705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls") pod "dns-default-9h5lq" (UID: "b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6") : secret "dns-default-metrics-tls" not found
Jul 02 13:45:51.100345 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:51.100299    2567 generic.go:356] "Generic (PLEG): container finished" podID="f24875be-dc41-4b03-84e6-e00f7c5d9df0" containerID="a2e676fd7e439a2e144ed6cd4cb32d9298c85104a18716b0b26c925b089cb310" exitCode=0
Jul 02 13:45:51.100522 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:51.100381    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n88db" event={"ID":"f24875be-dc41-4b03-84e6-e00f7c5d9df0","Type":"ContainerDied","Data":"a2e676fd7e439a2e144ed6cd4cb32d9298c85104a18716b0b26c925b089cb310"}
Jul 02 13:45:51.101853 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:51.101652    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kfvfm" event={"ID":"99d73dda-2da1-400e-bc32-a373ba20f546","Type":"ContainerStarted","Data":"f5c48c618809b16fae8840cddedd1f04995e51a9e638b56d4457c6633154f797"}
Jul 02 13:45:51.474041 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:51.473957    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:51.585550 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:51.585512    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:51.651548 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:51.651511    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:51.849407 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:51.849322    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:51.937809 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:51.937777    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:52.108436 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:52.108352    2567 generic.go:356] "Generic (PLEG): container finished" podID="f24875be-dc41-4b03-84e6-e00f7c5d9df0" containerID="678d5378c83f81a5613ed60b60ec0cfb3d0f2881a0e19683913b8548491df6a6" exitCode=0
Jul 02 13:45:52.108436 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:52.108419    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n88db" event={"ID":"f24875be-dc41-4b03-84e6-e00f7c5d9df0","Type":"ContainerDied","Data":"678d5378c83f81a5613ed60b60ec0cfb3d0f2881a0e19683913b8548491df6a6"}
Jul 02 13:45:53.033677 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:53.033434    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:53.033677 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:53.033583    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:45:53.033856 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:53.033720    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls podName:b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:57.033703865 +0000 UTC m=+42.538828347 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls") pod "dns-default-9h5lq" (UID: "b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6") : secret "dns-default-metrics-tls" not found
Jul 02 13:45:53.112296 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:53.112262    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kfvfm" event={"ID":"99d73dda-2da1-400e-bc32-a373ba20f546","Type":"ContainerStarted","Data":"2b71ad1a22e24012dad5e9964b0f55d478e5f140bf0a31494762222afe398ced"}
Jul 02 13:45:53.115542 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:53.115517    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n88db" event={"ID":"f24875be-dc41-4b03-84e6-e00f7c5d9df0","Type":"ContainerStarted","Data":"a1d2926d75d4ebfe721d629d931476ee9dfaed149d5197b7f31bde79c2d58c16"}
Jul 02 13:45:53.134900 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:53.134854    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kfvfm" podStartSLOduration=1.763233981 podStartE2EDuration="4.134839353s" totalImagesPullingTime="2.371605372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:53.133498445 +0000 UTC m=+38.638622940" watchObservedRunningTime="2026-07-02 13:45:53.134839353 +0000 UTC m=+38.639963835"
Jul 02 13:45:53.175732 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:53.175685    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n88db" podStartSLOduration=7.876130672 podStartE2EDuration="38.175669348s" totalImagesPullingTime="30.299538676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:53.174053052 +0000 UTC m=+38.679177555" watchObservedRunningTime="2026-07-02 13:45:53.175669348 +0000 UTC m=+38.680793851"
Jul 02 13:45:53.330285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:53.330208    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:53.512908 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:53.512876    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:53.578078 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:53.578049    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:53.762453 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:53.762371    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:53.902095 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:53.902063    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:45:57.061776 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:45:57.061733    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:45:57.062134 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:57.061888    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:45:57.062134 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:45:57.061956    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls podName:b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:05.061940434 +0000 UTC m=+50.567064916 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls") pod "dns-default-9h5lq" (UID: "b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6") : secret "dns-default-metrics-tls" not found
Jul 02 13:46:05.121260 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:05.121219    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:46:05.121743 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:05.121348    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:46:05.121743 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:05.121408    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls podName:b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:21.121394743 +0000 UTC m=+66.626519225 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls") pod "dns-default-9h5lq" (UID: "b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6") : secret "dns-default-metrics-tls" not found
Jul 02 13:46:12.510984 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:12.510950    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:14.070214 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:14.070176    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:14.099982 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:14.099957    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xtms2"
Jul 02 13:46:14.307256 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:14.307221    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:14.564383 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:14.564348    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:19.613838 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:19.613787    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:46:19.613838 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:19.613854    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:46:19.616598 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:19.616580    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 13:46:19.616647 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:19.616627    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 13:46:19.624715 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:19.624689    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 02 13:46:19.624777 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:19.624767    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs podName:d01be1dd-2bb1-409d-aaaa-6094d6e05f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:47:23.624742961 +0000 UTC m=+129.129867462 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs") pod "network-metrics-daemon-whrk8" (UID: "d01be1dd-2bb1-409d-aaaa-6094d6e05f26") : secret "metrics-daemon-secret" not found
Jul 02 13:46:19.628463 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:19.628441    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49ea922e-112c-480c-9a70-7992f0e29149-original-pull-secret\") pod \"global-pull-secret-syncer-jbfz7\" (UID: \"49ea922e-112c-480c-9a70-7992f0e29149\") " pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:46:19.714990 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:19.714943    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44gdt\" (UniqueName: \"kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt\") pod \"network-check-target-bt4j2\" (UID: \"afddd3e6-b476-40e8-95d8-2b2332236c91\") " pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:46:19.717765 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:19.717747    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 13:46:19.718753 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:19.718739    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbfz7"
Jul 02 13:46:19.728221 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:19.728199    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 13:46:19.739204 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:19.739172    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44gdt\" (UniqueName: \"kubernetes.io/projected/afddd3e6-b476-40e8-95d8-2b2332236c91-kube-api-access-44gdt\") pod \"network-check-target-bt4j2\" (UID: \"afddd3e6-b476-40e8-95d8-2b2332236c91\") " pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:46:19.842905 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:19.842870    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jbfz7"]
Jul 02 13:46:19.846214 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:46:19.846186    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49ea922e_112c_480c_9a70_7992f0e29149.slice/crio-9bfd476275f0b38db9951682109e877faa52386569d84f231a8a107636524bec WatchSource:0}: Error finding container 9bfd476275f0b38db9951682109e877faa52386569d84f231a8a107636524bec: Status 404 returned error can't find the container with id 9bfd476275f0b38db9951682109e877faa52386569d84f231a8a107636524bec
Jul 02 13:46:20.027757 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:20.027673    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7vs28\""
Jul 02 13:46:20.035185 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:20.035160    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:46:20.166044 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:20.166012    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bt4j2"]
Jul 02 13:46:20.168891 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:46:20.168865    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafddd3e6_b476_40e8_95d8_2b2332236c91.slice/crio-fd03a3b8cb5a0d2a0898a65788891cbaa8f34e30f99801a88d7742c6318b6d6d WatchSource:0}: Error finding container fd03a3b8cb5a0d2a0898a65788891cbaa8f34e30f99801a88d7742c6318b6d6d: Status 404 returned error can't find the container with id fd03a3b8cb5a0d2a0898a65788891cbaa8f34e30f99801a88d7742c6318b6d6d
Jul 02 13:46:20.177626 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:20.177600    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bt4j2" event={"ID":"afddd3e6-b476-40e8-95d8-2b2332236c91","Type":"ContainerStarted","Data":"fd03a3b8cb5a0d2a0898a65788891cbaa8f34e30f99801a88d7742c6318b6d6d"}
Jul 02 13:46:20.178474 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:20.178456    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jbfz7" event={"ID":"49ea922e-112c-480c-9a70-7992f0e29149","Type":"ContainerStarted","Data":"9bfd476275f0b38db9951682109e877faa52386569d84f231a8a107636524bec"}
Jul 02 13:46:21.129059 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:21.128999    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:46:21.129604 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:21.129142    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:46:21.129604 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:21.129199    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls podName:b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:53.129185181 +0000 UTC m=+98.634309662 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls") pod "dns-default-9h5lq" (UID: "b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6") : secret "dns-default-metrics-tls" not found
Jul 02 13:46:25.192571 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:25.192530    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jbfz7" event={"ID":"49ea922e-112c-480c-9a70-7992f0e29149","Type":"ContainerStarted","Data":"3655c5f453a6c436323c19c9f372681632525f8c8487cc945047427bb4aeac19"}
Jul 02 13:46:25.193942 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:25.193918    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bt4j2" event={"ID":"afddd3e6-b476-40e8-95d8-2b2332236c91","Type":"ContainerStarted","Data":"42b0f4235ff4ae9a2e0306660bb8334da88d94565bfe7f941b59718896060376"}
Jul 02 13:46:25.194058 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:25.194047    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:46:25.211700 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:25.211652    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jbfz7" podStartSLOduration=65.60791369 podStartE2EDuration="1m10.211638605s" totalImagesPullingTime="4.603724915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:25.210990477 +0000 UTC m=+70.716114981" watchObservedRunningTime="2026-07-02 13:46:25.211638605 +0000 UTC m=+70.716763107"
Jul 02 13:46:41.105026 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.104881    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bt4j2" podStartSLOduration=81.827540017 podStartE2EDuration="1m26.104847721s" totalImagesPullingTime="4.277307704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:25.228934249 +0000 UTC m=+70.734058753" watchObservedRunningTime="2026-07-02 13:46:41.104847721 +0000 UTC m=+86.609972206"
Jul 02 13:46:41.105980 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.105960    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5f67b55555-z4sj4"]
Jul 02 13:46:41.108542 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.108521    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.113137 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.113109    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 02 13:46:41.113820 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.113790    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-jmppj\""
Jul 02 13:46:41.113937 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.113820    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 02 13:46:41.113937 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.113829    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 02 13:46:41.113937 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.113856    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 02 13:46:41.121118 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.121091    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 02 13:46:41.121814 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.121792    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-z4sj4"]
Jul 02 13:46:41.158280 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.158243    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75095516-19ab-4843-aeb8-8b4df9fc1005-tmp\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.158280 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.158282    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75095516-19ab-4843-aeb8-8b4df9fc1005-serving-cert\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.158555 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.158304    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/75095516-19ab-4843-aeb8-8b4df9fc1005-snapshots\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.158555 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.158331    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c59j6\" (UniqueName: \"kubernetes.io/projected/75095516-19ab-4843-aeb8-8b4df9fc1005-kube-api-access-c59j6\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.158555 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.158413    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75095516-19ab-4843-aeb8-8b4df9fc1005-service-ca-bundle\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.158555 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.158455    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75095516-19ab-4843-aeb8-8b4df9fc1005-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.258815 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.258777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/75095516-19ab-4843-aeb8-8b4df9fc1005-snapshots\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.258815 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.258823    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c59j6\" (UniqueName: \"kubernetes.io/projected/75095516-19ab-4843-aeb8-8b4df9fc1005-kube-api-access-c59j6\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.259062 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.258856    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75095516-19ab-4843-aeb8-8b4df9fc1005-service-ca-bundle\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.259062 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.258879    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75095516-19ab-4843-aeb8-8b4df9fc1005-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.259062 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.258910    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75095516-19ab-4843-aeb8-8b4df9fc1005-tmp\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.259062 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.258933    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75095516-19ab-4843-aeb8-8b4df9fc1005-serving-cert\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.259501 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.259462    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/75095516-19ab-4843-aeb8-8b4df9fc1005-snapshots\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.259605 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.259462    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75095516-19ab-4843-aeb8-8b4df9fc1005-tmp\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.259605 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.259543    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75095516-19ab-4843-aeb8-8b4df9fc1005-service-ca-bundle\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.259967 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.259946    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75095516-19ab-4843-aeb8-8b4df9fc1005-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.261205 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.261185    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75095516-19ab-4843-aeb8-8b4df9fc1005-serving-cert\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.272349 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.272318    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c59j6\" (UniqueName: \"kubernetes.io/projected/75095516-19ab-4843-aeb8-8b4df9fc1005-kube-api-access-c59j6\") pod \"insights-operator-5f67b55555-z4sj4\" (UID: \"75095516-19ab-4843-aeb8-8b4df9fc1005\") " pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.320390 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.320355    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"]
Jul 02 13:46:41.323038 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.323011    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.323170 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.323020    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6c987749f-vpfgn"]
Jul 02 13:46:41.324951 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.324928    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.327001 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.326973    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 02 13:46:41.327143 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.327055    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 02 13:46:41.327227 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.327169    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 02 13:46:41.327673 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.327655    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 02 13:46:41.327888 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.327874    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kl44z\""
Jul 02 13:46:41.327888 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.327883    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 02 13:46:41.328022 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.327904    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 02 13:46:41.328022 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.327904    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-v75rb\""
Jul 02 13:46:41.332363 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.332348    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 02 13:46:41.333746 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.333729    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 02 13:46:41.334111 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.334097    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 02 13:46:41.334309 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.334271    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 02 13:46:41.337097 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.337071    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 02 13:46:41.340935 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.340916    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"]
Jul 02 13:46:41.344455 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.344424    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c987749f-vpfgn"]
Jul 02 13:46:41.359935 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.359849    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.359935 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.359886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc379a40-405c-417d-bb40-ab9064a2b14b-serving-cert\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.359935 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.359906    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4ccc\" (UniqueName: \"kubernetes.io/projected/fc379a40-405c-417d-bb40-ab9064a2b14b-kube-api-access-c4ccc\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.360165 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.359965    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.360165 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.359994    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-stats-auth\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.360165 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.360020    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmvdx\" (UniqueName: \"kubernetes.io/projected/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-kube-api-access-mmvdx\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.360165 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.360039    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc379a40-405c-417d-bb40-ab9064a2b14b-trusted-ca\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.360165 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.360109    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-default-certificate\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.360165 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.360124    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc379a40-405c-417d-bb40-ab9064a2b14b-config\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.417607 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.417571    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-z4sj4"
Jul 02 13:46:41.460785 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.460752    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-default-certificate\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.460785 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.460788    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc379a40-405c-417d-bb40-ab9064a2b14b-config\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.461014 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.460825    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.461014 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.460840    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc379a40-405c-417d-bb40-ab9064a2b14b-serving-cert\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.461014 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.460859    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4ccc\" (UniqueName: \"kubernetes.io/projected/fc379a40-405c-417d-bb40-ab9064a2b14b-kube-api-access-c4ccc\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.461014 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.460884    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.461014 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.460907    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-stats-auth\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.461014 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.460931    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmvdx\" (UniqueName: \"kubernetes.io/projected/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-kube-api-access-mmvdx\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.461014 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.460953    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc379a40-405c-417d-bb40-ab9064a2b14b-trusted-ca\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.461810 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:41.461570    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 02 13:46:41.461810 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:41.461608    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle podName:c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:41.961585388 +0000 UTC m=+87.466709886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle") pod "router-default-6c987749f-vpfgn" (UID: "c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7") : configmap references non-existent config key: service-ca.crt
Jul 02 13:46:41.461810 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:41.461641    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs podName:c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:41.961622498 +0000 UTC m=+87.466746980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs") pod "router-default-6c987749f-vpfgn" (UID: "c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7") : secret "router-metrics-certs-default" not found
Jul 02 13:46:41.462382 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.462107    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc379a40-405c-417d-bb40-ab9064a2b14b-config\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.462382 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.462341    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc379a40-405c-417d-bb40-ab9064a2b14b-trusted-ca\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.464473 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.464429    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-default-certificate\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.465977 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.465942    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-stats-auth\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.466100 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.466079    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc379a40-405c-417d-bb40-ab9064a2b14b-serving-cert\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.477120 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.477071    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4ccc\" (UniqueName: \"kubernetes.io/projected/fc379a40-405c-417d-bb40-ab9064a2b14b-kube-api-access-c4ccc\") pod \"console-operator-7fdbdf9c99-jjmnl\" (UID: \"fc379a40-405c-417d-bb40-ab9064a2b14b\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.477521 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.477500    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmvdx\" (UniqueName: \"kubernetes.io/projected/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-kube-api-access-mmvdx\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.508372 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.508297    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:41.543006 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.542976    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-z4sj4"]
Jul 02 13:46:41.546403 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:46:41.546372    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75095516_19ab_4843_aeb8_8b4df9fc1005.slice/crio-c941d5e379e14ea167d5ea33dfdd38cc43dbbdcdab9135002ee20d3b8ab0ac1d WatchSource:0}: Error finding container c941d5e379e14ea167d5ea33dfdd38cc43dbbdcdab9135002ee20d3b8ab0ac1d: Status 404 returned error can't find the container with id c941d5e379e14ea167d5ea33dfdd38cc43dbbdcdab9135002ee20d3b8ab0ac1d
Jul 02 13:46:41.605945 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.605910    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:41.634045 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.633973    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:41.764840 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.764811    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"]
Jul 02 13:46:41.767615 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:46:41.767588    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc379a40_405c_417d_bb40_ab9064a2b14b.slice/crio-af0b1936e4ce7fc7921035458cfaa48e2929ebb4fc9e29f27b78ad6b189c6f0d WatchSource:0}: Error finding container af0b1936e4ce7fc7921035458cfaa48e2929ebb4fc9e29f27b78ad6b189c6f0d: Status 404 returned error can't find the container with id af0b1936e4ce7fc7921035458cfaa48e2929ebb4fc9e29f27b78ad6b189c6f0d
Jul 02 13:46:41.965121 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.965026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.965121 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:41.965080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:41.965297 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:41.965165    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 02 13:46:41.965297 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:41.965202    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle podName:c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:42.965189069 +0000 UTC m=+88.470313551 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle") pod "router-default-6c987749f-vpfgn" (UID: "c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7") : configmap references non-existent config key: service-ca.crt
Jul 02 13:46:41.965297 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:41.965217    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs podName:c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:42.965210795 +0000 UTC m=+88.470335277 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs") pod "router-default-6c987749f-vpfgn" (UID: "c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7") : secret "router-metrics-certs-default" not found
Jul 02 13:46:42.240620 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:42.240526    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl" event={"ID":"fc379a40-405c-417d-bb40-ab9064a2b14b","Type":"ContainerStarted","Data":"af0b1936e4ce7fc7921035458cfaa48e2929ebb4fc9e29f27b78ad6b189c6f0d"}
Jul 02 13:46:42.241804 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:42.241767    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-z4sj4" event={"ID":"75095516-19ab-4843-aeb8-8b4df9fc1005","Type":"ContainerStarted","Data":"c941d5e379e14ea167d5ea33dfdd38cc43dbbdcdab9135002ee20d3b8ab0ac1d"}
Jul 02 13:46:42.972208 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:42.972168    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:42.972456 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:42.972229    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:42.972456 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:42.972347    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 02 13:46:42.972456 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:42.972374    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle podName:c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:44.97235513 +0000 UTC m=+90.477479626 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle") pod "router-default-6c987749f-vpfgn" (UID: "c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7") : configmap references non-existent config key: service-ca.crt
Jul 02 13:46:42.972456 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:42.972409    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs podName:c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:44.972392106 +0000 UTC m=+90.477516588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs") pod "router-default-6c987749f-vpfgn" (UID: "c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7") : secret "router-metrics-certs-default" not found
Jul 02 13:46:43.586161 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:43.586115    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:44.988840 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:44.988779    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:44.989277 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:44.988866    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:44.989277 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:44.988926    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 02 13:46:44.989277 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:44.988999    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs podName:c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:48.988982964 +0000 UTC m=+94.494107448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs") pod "router-default-6c987749f-vpfgn" (UID: "c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7") : secret "router-metrics-certs-default" not found
Jul 02 13:46:44.989277 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:44.989034    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle podName:c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:48.989017247 +0000 UTC m=+94.494141743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle") pod "router-default-6c987749f-vpfgn" (UID: "c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7") : configmap references non-existent config key: service-ca.crt
Jul 02 13:46:45.238653 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.238621    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nxgcc"]
Jul 02 13:46:45.241758 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.241690    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.249471 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.249450    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 02 13:46:45.252606 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.252586    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/0.log"
Jul 02 13:46:45.252712 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.252622    2567 generic.go:356] "Generic (PLEG): container finished" podID="fc379a40-405c-417d-bb40-ab9064a2b14b" containerID="095a6a21420bfb922cf57c2244b5a2491a88f71ceac962d705bf0a1d486e5688" exitCode=255
Jul 02 13:46:45.252712 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.252675    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl" event={"ID":"fc379a40-405c-417d-bb40-ab9064a2b14b","Type":"ContainerDied","Data":"095a6a21420bfb922cf57c2244b5a2491a88f71ceac962d705bf0a1d486e5688"}
Jul 02 13:46:45.252858 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.252830    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 02 13:46:45.252911 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.252844    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmzxj\""
Jul 02 13:46:45.252999 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.252967    2567 scope.go:117] "RemoveContainer" containerID="095a6a21420bfb922cf57c2244b5a2491a88f71ceac962d705bf0a1d486e5688"
Jul 02 13:46:45.254269 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.254248    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-z4sj4" event={"ID":"75095516-19ab-4843-aeb8-8b4df9fc1005","Type":"ContainerStarted","Data":"207099eb6e6133785da15fc35dfffed158ac3c7bafa3f37420862a6558d3c330"}
Jul 02 13:46:45.291023 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.291000    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d753af15-bfcf-4f37-8d5e-d894521e6e5b-data-volume\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.291162 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.291072    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.291162 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.291119    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsbq7\" (UniqueName: \"kubernetes.io/projected/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-api-access-bsbq7\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.291422 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.291392    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d753af15-bfcf-4f37-8d5e-d894521e6e5b-crio-socket\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.291510 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.291455    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.295319 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.295290    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nxgcc"]
Jul 02 13:46:45.362894 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.362867    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:45.368646 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.368591    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5f67b55555-z4sj4" podStartSLOduration=1.693820718 podStartE2EDuration="4.368572221s" totalImagesPullingTime="2.674751503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:45.347785619 +0000 UTC m=+90.852910134" watchObservedRunningTime="2026-07-02 13:46:45.368572221 +0000 UTC m=+90.873696725"
Jul 02 13:46:45.392340 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.392311    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d753af15-bfcf-4f37-8d5e-d894521e6e5b-crio-socket\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.392518 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.392347    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.392518 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.392380    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d753af15-bfcf-4f37-8d5e-d894521e6e5b-data-volume\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.392518 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.392423    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d753af15-bfcf-4f37-8d5e-d894521e6e5b-crio-socket\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.392518 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.392437    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.392518 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:45.392507    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:46:45.392518 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:45.392517    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:46:45.392809 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:45.392576    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm podName:d753af15-bfcf-4f37-8d5e-d894521e6e5b nodeName:}" failed. No retries permitted until 2026-07-02 13:46:45.892557169 +0000 UTC m=+91.397681664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nxgcc" (UID: "d753af15-bfcf-4f37-8d5e-d894521e6e5b") : configmap "kube-rbac-proxy" not found
Jul 02 13:46:45.392809 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.392508    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsbq7\" (UniqueName: \"kubernetes.io/projected/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-api-access-bsbq7\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.392809 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:45.392592    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls podName:d753af15-bfcf-4f37-8d5e-d894521e6e5b nodeName:}" failed. No retries permitted until 2026-07-02 13:46:45.892583265 +0000 UTC m=+91.397707747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nxgcc" (UID: "d753af15-bfcf-4f37-8d5e-d894521e6e5b") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:46:45.392809 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.392731    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d753af15-bfcf-4f37-8d5e-d894521e6e5b-data-volume\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.403252 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.403226    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsbq7\" (UniqueName: \"kubernetes.io/projected/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-api-access-bsbq7\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.432388 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.432357    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:45.700880 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.700848    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:45.792504 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.792458    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:45.896418 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.896380    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.896609 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.896433    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:45.896609 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:45.896538    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:46:45.896609 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:45.896560    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:46:45.896609 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:45.896591    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm podName:d753af15-bfcf-4f37-8d5e-d894521e6e5b nodeName:}" failed. No retries permitted until 2026-07-02 13:46:46.896577129 +0000 UTC m=+92.401701611 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nxgcc" (UID: "d753af15-bfcf-4f37-8d5e-d894521e6e5b") : configmap "kube-rbac-proxy" not found
Jul 02 13:46:45.896755 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:45.896617    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls podName:d753af15-bfcf-4f37-8d5e-d894521e6e5b nodeName:}" failed. No retries permitted until 2026-07-02 13:46:46.896603375 +0000 UTC m=+92.401727856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nxgcc" (UID: "d753af15-bfcf-4f37-8d5e-d894521e6e5b") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:46:45.966908 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:45.966830    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:46.069494 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.069458    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:46.195537 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.195465    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:46.259394 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.259317    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 13:46:46.259786 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.259770    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/0.log"
Jul 02 13:46:46.259843 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.259809    2567 generic.go:356] "Generic (PLEG): container finished" podID="fc379a40-405c-417d-bb40-ab9064a2b14b" containerID="19d332bdf706a42ea48da91a5995b46e8f8c9d83d264bdf3b85ca2b9a23213bc" exitCode=255
Jul 02 13:46:46.259906 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.259885    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl" event={"ID":"fc379a40-405c-417d-bb40-ab9064a2b14b","Type":"ContainerDied","Data":"19d332bdf706a42ea48da91a5995b46e8f8c9d83d264bdf3b85ca2b9a23213bc"}
Jul 02 13:46:46.259980 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.259968    2567 scope.go:117] "RemoveContainer" containerID="095a6a21420bfb922cf57c2244b5a2491a88f71ceac962d705bf0a1d486e5688"
Jul 02 13:46:46.260256 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.260242    2567 scope.go:117] "RemoveContainer" containerID="19d332bdf706a42ea48da91a5995b46e8f8c9d83d264bdf3b85ca2b9a23213bc"
Jul 02 13:46:46.260428 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:46.260406    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-jjmnl_openshift-console-operator(fc379a40-405c-417d-bb40-ab9064a2b14b)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl" podUID="fc379a40-405c-417d-bb40-ab9064a2b14b"
Jul 02 13:46:46.283498 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.283455    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:46.709067 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.709034    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:46.804121 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.804092    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:46.871080 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.871050    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:46.904451 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.904424    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:46.904615 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:46.904472    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:46.904615 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:46.904584    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:46:46.904695 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:46.904642    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls podName:d753af15-bfcf-4f37-8d5e-d894521e6e5b nodeName:}" failed. No retries permitted until 2026-07-02 13:46:48.904629231 +0000 UTC m=+94.409753716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nxgcc" (UID: "d753af15-bfcf-4f37-8d5e-d894521e6e5b") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:46:46.904695 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:46.904584    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:46:46.904761 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:46.904736    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm podName:d753af15-bfcf-4f37-8d5e-d894521e6e5b nodeName:}" failed. No retries permitted until 2026-07-02 13:46:48.904723982 +0000 UTC m=+94.409848483 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nxgcc" (UID: "d753af15-bfcf-4f37-8d5e-d894521e6e5b") : configmap "kube-rbac-proxy" not found
Jul 02 13:46:47.264383 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:47.264355    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 13:46:47.264795 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:47.264680    2567 scope.go:117] "RemoveContainer" containerID="19d332bdf706a42ea48da91a5995b46e8f8c9d83d264bdf3b85ca2b9a23213bc"
Jul 02 13:46:47.264842 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:47.264823    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-jjmnl_openshift-console-operator(fc379a40-405c-417d-bb40-ab9064a2b14b)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl" podUID="fc379a40-405c-417d-bb40-ab9064a2b14b"
Jul 02 13:46:47.450465 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:47.450443    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zqz2f_747984cc-ab52-45dc-99ef-53c93e5bd101/dns-node-resolver/0.log"
Jul 02 13:46:48.256521 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.256456    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f8rvf_110466f9-e7b0-4a07-8a64-126c9ebaf9c1/node-ca/0.log"
Jul 02 13:46:48.276259 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.276223    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"]
Jul 02 13:46:48.280247 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.280231    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"
Jul 02 13:46:48.286520 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.286500    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 02 13:46:48.286771 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.286757    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 02 13:46:48.287162 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.287150    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9gdhz\""
Jul 02 13:46:48.287382 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.287366    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 02 13:46:48.287432 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.287388    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 02 13:46:48.294442 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.294421    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"]
Jul 02 13:46:48.314668 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.314628    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdczl\" (UniqueName: \"kubernetes.io/projected/827302c6-2024-413c-8571-4bf300d3ba9f-kube-api-access-zdczl\") pod \"service-ca-7bb4f4bcc7-b9np9\" (UID: \"827302c6-2024-413c-8571-4bf300d3ba9f\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"
Jul 02 13:46:48.314811 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.314688    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/827302c6-2024-413c-8571-4bf300d3ba9f-signing-key\") pod \"service-ca-7bb4f4bcc7-b9np9\" (UID: \"827302c6-2024-413c-8571-4bf300d3ba9f\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"
Jul 02 13:46:48.314811 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.314706    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/827302c6-2024-413c-8571-4bf300d3ba9f-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-b9np9\" (UID: \"827302c6-2024-413c-8571-4bf300d3ba9f\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"
Jul 02 13:46:48.346526 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.346499    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:48.415666 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.415622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdczl\" (UniqueName: \"kubernetes.io/projected/827302c6-2024-413c-8571-4bf300d3ba9f-kube-api-access-zdczl\") pod \"service-ca-7bb4f4bcc7-b9np9\" (UID: \"827302c6-2024-413c-8571-4bf300d3ba9f\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"
Jul 02 13:46:48.415830 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.415684    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/827302c6-2024-413c-8571-4bf300d3ba9f-signing-key\") pod \"service-ca-7bb4f4bcc7-b9np9\" (UID: \"827302c6-2024-413c-8571-4bf300d3ba9f\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"
Jul 02 13:46:48.415830 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.415702    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/827302c6-2024-413c-8571-4bf300d3ba9f-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-b9np9\" (UID: \"827302c6-2024-413c-8571-4bf300d3ba9f\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"
Jul 02 13:46:48.416262 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.416243    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/827302c6-2024-413c-8571-4bf300d3ba9f-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-b9np9\" (UID: \"827302c6-2024-413c-8571-4bf300d3ba9f\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"
Jul 02 13:46:48.418177 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.418146    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/827302c6-2024-413c-8571-4bf300d3ba9f-signing-key\") pod \"service-ca-7bb4f4bcc7-b9np9\" (UID: \"827302c6-2024-413c-8571-4bf300d3ba9f\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"
Jul 02 13:46:48.425294 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.425272    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdczl\" (UniqueName: \"kubernetes.io/projected/827302c6-2024-413c-8571-4bf300d3ba9f-kube-api-access-zdczl\") pod \"service-ca-7bb4f4bcc7-b9np9\" (UID: \"827302c6-2024-413c-8571-4bf300d3ba9f\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"
Jul 02 13:46:48.447278 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.447251    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:48.588938 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.588903    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"
Jul 02 13:46:48.590013 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.589988    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:48.690735 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.690702    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:48.712013 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.711984    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-b9np9"]
Jul 02 13:46:48.714762 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:46:48.714729    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod827302c6_2024_413c_8571_4bf300d3ba9f.slice/crio-c36a488607158d91a22a813332c698d81e33ed413da68de36f5a6c0e212e187d WatchSource:0}: Error finding container c36a488607158d91a22a813332c698d81e33ed413da68de36f5a6c0e212e187d: Status 404 returned error can't find the container with id c36a488607158d91a22a813332c698d81e33ed413da68de36f5a6c0e212e187d
Jul 02 13:46:48.851367 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.851287    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kfvfm_99d73dda-2da1-400e-bc32-a373ba20f546/serve-healthcheck-canary/0.log"
Jul 02 13:46:48.919464 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.919431    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:48.919673 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:48.919505    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:48.919673 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:48.919589    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:46:48.919673 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:48.919610    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:46:48.919673 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:48.919636    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm podName:d753af15-bfcf-4f37-8d5e-d894521e6e5b nodeName:}" failed. No retries permitted until 2026-07-02 13:46:52.919623055 +0000 UTC m=+98.424747537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nxgcc" (UID: "d753af15-bfcf-4f37-8d5e-d894521e6e5b") : configmap "kube-rbac-proxy" not found
Jul 02 13:46:48.919673 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:48.919670    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls podName:d753af15-bfcf-4f37-8d5e-d894521e6e5b nodeName:}" failed. No retries permitted until 2026-07-02 13:46:52.919652187 +0000 UTC m=+98.424776685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nxgcc" (UID: "d753af15-bfcf-4f37-8d5e-d894521e6e5b") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:46:49.020587 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:49.020547    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:49.020587 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:49.020594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:49.020782 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:49.020723    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 02 13:46:49.020782 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:49.020762    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle podName:c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:57.020744899 +0000 UTC m=+102.525869396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle") pod "router-default-6c987749f-vpfgn" (UID: "c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7") : configmap references non-existent config key: service-ca.crt
Jul 02 13:46:49.020782 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:49.020780    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs podName:c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:57.020773316 +0000 UTC m=+102.525897797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs") pod "router-default-6c987749f-vpfgn" (UID: "c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7") : secret "router-metrics-certs-default" not found
Jul 02 13:46:49.272158 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:49.272074    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9" event={"ID":"827302c6-2024-413c-8571-4bf300d3ba9f","Type":"ContainerStarted","Data":"c36a488607158d91a22a813332c698d81e33ed413da68de36f5a6c0e212e187d"}
Jul 02 13:46:49.386611 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:49.386559    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:51.283303 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:51.283268    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9" event={"ID":"827302c6-2024-413c-8571-4bf300d3ba9f","Type":"ContainerStarted","Data":"90bb4153d088ec9dee4575cb14053b430fe99991281f279b1acf71ef53e3b5ee"}
Jul 02 13:46:51.312592 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:51.312534    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7bb4f4bcc7-b9np9" podStartSLOduration=1.349502513 podStartE2EDuration="3.312516415s" totalImagesPullingTime="1.963013902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:46:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:51.30777772 +0000 UTC m=+96.812902222" watchObservedRunningTime="2026-07-02 13:46:51.312516415 +0000 UTC m=+96.817640918"
Jul 02 13:46:51.634695 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:51.634664    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:51.634851 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:51.634715    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:46:51.635053 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:51.635041    2567 scope.go:117] "RemoveContainer" containerID="19d332bdf706a42ea48da91a5995b46e8f8c9d83d264bdf3b85ca2b9a23213bc"
Jul 02 13:46:51.635195 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:51.635177    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-jjmnl_openshift-console-operator(fc379a40-405c-417d-bb40-ab9064a2b14b)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl" podUID="fc379a40-405c-417d-bb40-ab9064a2b14b"
Jul 02 13:46:52.030077 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:52.029974    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:52.264791 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:52.264749    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:52.425918 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:52.425873    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:52.587477 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:52.587421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:52.739804 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:52.739692    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:52.957969 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:52.957935    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:52.958133 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:52.958067    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:46:52.958133 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:52.958075    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:46:52.958133 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:52.958126    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm podName:d753af15-bfcf-4f37-8d5e-d894521e6e5b nodeName:}" failed. No retries permitted until 2026-07-02 13:47:00.958109479 +0000 UTC m=+106.463233961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nxgcc" (UID: "d753af15-bfcf-4f37-8d5e-d894521e6e5b") : configmap "kube-rbac-proxy" not found
Jul 02 13:46:52.958327 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:52.958188    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:46:52.958327 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:52.958248    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls podName:d753af15-bfcf-4f37-8d5e-d894521e6e5b nodeName:}" failed. No retries permitted until 2026-07-02 13:47:00.958232328 +0000 UTC m=+106.463356830 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nxgcc" (UID: "d753af15-bfcf-4f37-8d5e-d894521e6e5b") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:46:53.159110 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:53.159083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:46:53.159259 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:53.159242    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:46:53.159314 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:53.159305    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls podName:b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:47:57.159289418 +0000 UTC m=+162.664413900 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls") pod "dns-default-9h5lq" (UID: "b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6") : secret "dns-default-metrics-tls" not found
Jul 02 13:46:53.444002 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:53.443908    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:53.578352 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:53.578299    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:54.018891 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:54.018860    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:54.255233 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:54.255192    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:54.422665 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:54.422627    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:54.848819 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:54.848783    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:55.588164 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:55.588118    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:55.683738 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:55.683705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:55.800911 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:55.800876    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:56.007575 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:56.007477    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:56.198989 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:56.198959    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bt4j2"
Jul 02 13:46:56.200022 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:56.199985    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:56.318508 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:56.318459    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:56.434620 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:56.434581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:56.520269 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:56.520238    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:56.608820 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:56.608734    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:57.097405 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:57.097371    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:57.097896 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:57.097415    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:57.097896 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:46:57.097604    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle podName:c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7 nodeName:}" failed. No retries permitted until 2026-07-02 13:47:13.097583095 +0000 UTC m=+118.602707578 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle") pod "router-default-6c987749f-vpfgn" (UID: "c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7") : configmap references non-existent config key: service-ca.crt
Jul 02 13:46:57.099958 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:57.099932    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-metrics-certs\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:46:57.610237 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:57.610167    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:57.948673 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:57.948555    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:58.048464 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:58.048417    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:58.207795 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:58.207686    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:58.384407 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:58.384353    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:58.477290 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:58.477192    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:58.607941 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:58.607909    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:46:59.799890 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:46:59.799861    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:00.006472 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:00.006433    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:00.191866 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:00.191783    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:00.403227 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:00.403180    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:00.619838 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:00.619808    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:01.031935 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:01.031850    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:47:01.031935 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:01.031898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:47:01.032329 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:47:01.032007    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:47:01.032329 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:47:01.032071    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls podName:d753af15-bfcf-4f37-8d5e-d894521e6e5b nodeName:}" failed. No retries permitted until 2026-07-02 13:47:17.032055467 +0000 UTC m=+122.537179953 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nxgcc" (UID: "d753af15-bfcf-4f37-8d5e-d894521e6e5b") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:47:01.032455 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:01.032435    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d753af15-bfcf-4f37-8d5e-d894521e6e5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:47:01.603615 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:01.603586    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:01.997468 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:01.997387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:02.203096 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:02.203055    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:02.396586 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:02.396551    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:02.703155 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:02.703075    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:03.898587 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:03.898557    2567 scope.go:117] "RemoveContainer" containerID="19d332bdf706a42ea48da91a5995b46e8f8c9d83d264bdf3b85ca2b9a23213bc"
Jul 02 13:47:04.322297 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:04.322268    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 13:47:04.322513 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:04.322351    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl" event={"ID":"fc379a40-405c-417d-bb40-ab9064a2b14b","Type":"ContainerStarted","Data":"9fd2fd5c2dbac4a0812733879855d3f11d1495889d37b5a6fbd80caca0db788b"}
Jul 02 13:47:04.322742 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:04.322716    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:47:04.343314 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:04.343266    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl" podStartSLOduration=20.889225761 podStartE2EDuration="23.343250609s" totalImagesPullingTime="2.454024848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:04.341375432 +0000 UTC m=+109.846499949" watchObservedRunningTime="2026-07-02 13:47:04.343250609 +0000 UTC m=+109.848375112"
Jul 02 13:47:04.411455 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:04.411428    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-jjmnl"
Jul 02 13:47:04.617979 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:04.617895    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:04.718584 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:04.718543    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:04.783416 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:04.783380    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:04.913991 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:04.913917    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:04.975011 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:04.974981    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:05.078297 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:05.078253    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:05.179329 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:05.179246    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:05.472595 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:05.472516    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:05.541140 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:05.541103    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:05.687072 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:05.687037    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:05.764511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:05.764407    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:06.292820 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:06.292787    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:06.501451 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:06.501414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:06.666316 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:06.666283    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:07.117274 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:07.117237    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:07.195380 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:07.195350    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:13.127059 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.127026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:47:13.127633 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.127612    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7-service-ca-bundle\") pod \"router-default-6c987749f-vpfgn\" (UID: \"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7\") " pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:47:13.139114 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.139082    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:47:13.272608 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.272582    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c987749f-vpfgn"]
Jul 02 13:47:13.275077 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:47:13.275043    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b71b52_1d0a_4b4e_98d7_6a1e9fcfdac7.slice/crio-1461b7d972ae48356f0590c268ec11db4a55f2ba7cad3134ba380ea90aa60f36 WatchSource:0}: Error finding container 1461b7d972ae48356f0590c268ec11db4a55f2ba7cad3134ba380ea90aa60f36: Status 404 returned error can't find the container with id 1461b7d972ae48356f0590c268ec11db4a55f2ba7cad3134ba380ea90aa60f36
Jul 02 13:47:13.351138 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.351100    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c987749f-vpfgn" event={"ID":"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7","Type":"ContainerStarted","Data":"a2608246440c1976f4476cfcff95fba7db5f3e60ac8ff72e6d361a5495721647"}
Jul 02 13:47:13.351297 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.351157    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c987749f-vpfgn" event={"ID":"c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7","Type":"ContainerStarted","Data":"1461b7d972ae48356f0590c268ec11db4a55f2ba7cad3134ba380ea90aa60f36"}
Jul 02 13:47:13.396034 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.395932    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6c987749f-vpfgn" podStartSLOduration=32.395915302 podStartE2EDuration="32.395915302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:13.394866167 +0000 UTC m=+118.899990692" watchObservedRunningTime="2026-07-02 13:47:13.395915302 +0000 UTC m=+118.901039806"
Jul 02 13:47:13.471852 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.471814    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf"]
Jul 02 13:47:13.474091 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.474061    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf"
Jul 02 13:47:13.478621 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.478596    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-rkvpm\""
Jul 02 13:47:13.481419 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.481394    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 02 13:47:13.481565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.481451    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 02 13:47:13.481565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.481464    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 02 13:47:13.481565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.481471    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 02 13:47:13.515105 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.515076    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf"]
Jul 02 13:47:13.529680 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.529643    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:13.565420 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.565385    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5499b6fb8f-fnjd9"]
Jul 02 13:47:13.567197 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.567182    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-fnjd9"
Jul 02 13:47:13.571565 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.571547    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 02 13:47:13.571681 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.571628    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 02 13:47:13.571743 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.571701    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gzdnk\""
Jul 02 13:47:13.599344 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.599312    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-fnjd9"]
Jul 02 13:47:13.631618 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.631584    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d16a0359-1dc8-426e-bcdb-6f4143196fcd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84d699996d-p4cvf\" (UID: \"d16a0359-1dc8-426e-bcdb-6f4143196fcd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf"
Jul 02 13:47:13.631784 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.631693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t87m2\" (UniqueName: \"kubernetes.io/projected/d16a0359-1dc8-426e-bcdb-6f4143196fcd-kube-api-access-t87m2\") pod \"managed-serviceaccount-addon-agent-84d699996d-p4cvf\" (UID: \"d16a0359-1dc8-426e-bcdb-6f4143196fcd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf"
Jul 02 13:47:13.701038 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.700957    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-644d865c6b-mk8jh"]
Jul 02 13:47:13.702967 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.702945    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.704387 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.704364    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:13.709102 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.709082    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 02 13:47:13.709234 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.709081    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 02 13:47:13.709234 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.709217    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dkwdr\""
Jul 02 13:47:13.710064 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.710044    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 02 13:47:13.716125 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.716108    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 02 13:47:13.727145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.727124    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-644d865c6b-mk8jh"]
Jul 02 13:47:13.733028 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.733002    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t87m2\" (UniqueName: \"kubernetes.io/projected/d16a0359-1dc8-426e-bcdb-6f4143196fcd-kube-api-access-t87m2\") pod \"managed-serviceaccount-addon-agent-84d699996d-p4cvf\" (UID: \"d16a0359-1dc8-426e-bcdb-6f4143196fcd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf"
Jul 02 13:47:13.733148 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.733049    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d16a0359-1dc8-426e-bcdb-6f4143196fcd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84d699996d-p4cvf\" (UID: \"d16a0359-1dc8-426e-bcdb-6f4143196fcd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf"
Jul 02 13:47:13.733203 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.733184    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f26gf\" (UniqueName: \"kubernetes.io/projected/1ae29235-cca2-4a37-80ae-65fef194d3fc-kube-api-access-f26gf\") pod \"downloads-5499b6fb8f-fnjd9\" (UID: \"1ae29235-cca2-4a37-80ae-65fef194d3fc\") " pod="openshift-console/downloads-5499b6fb8f-fnjd9"
Jul 02 13:47:13.735503 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.735446    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d16a0359-1dc8-426e-bcdb-6f4143196fcd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84d699996d-p4cvf\" (UID: \"d16a0359-1dc8-426e-bcdb-6f4143196fcd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf"
Jul 02 13:47:13.750399 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.750378    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t87m2\" (UniqueName: \"kubernetes.io/projected/d16a0359-1dc8-426e-bcdb-6f4143196fcd-kube-api-access-t87m2\") pod \"managed-serviceaccount-addon-agent-84d699996d-p4cvf\" (UID: \"d16a0359-1dc8-426e-bcdb-6f4143196fcd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf"
Jul 02 13:47:13.799597 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.799534    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf"
Jul 02 13:47:13.834454 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.834412    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/585ed721-7d68-462d-9431-17135ff5475b-registry-tls\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.834640 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.834461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/585ed721-7d68-462d-9431-17135ff5475b-registry-certificates\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.834640 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.834496    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/585ed721-7d68-462d-9431-17135ff5475b-trusted-ca\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.834640 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.834608    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/585ed721-7d68-462d-9431-17135ff5475b-image-registry-private-configuration\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.834779 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.834658    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/585ed721-7d68-462d-9431-17135ff5475b-ca-trust-extracted\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.834779 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.834719    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/585ed721-7d68-462d-9431-17135ff5475b-bound-sa-token\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.834871 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.834794    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/585ed721-7d68-462d-9431-17135ff5475b-installation-pull-secrets\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.834871 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.834834    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f26gf\" (UniqueName: \"kubernetes.io/projected/1ae29235-cca2-4a37-80ae-65fef194d3fc-kube-api-access-f26gf\") pod \"downloads-5499b6fb8f-fnjd9\" (UID: \"1ae29235-cca2-4a37-80ae-65fef194d3fc\") " pod="openshift-console/downloads-5499b6fb8f-fnjd9"
Jul 02 13:47:13.834956 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.834875    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmg9w\" (UniqueName: \"kubernetes.io/projected/585ed721-7d68-462d-9431-17135ff5475b-kube-api-access-nmg9w\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.840250 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.840225    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:13.849961 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.849931    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f26gf\" (UniqueName: \"kubernetes.io/projected/1ae29235-cca2-4a37-80ae-65fef194d3fc-kube-api-access-f26gf\") pod \"downloads-5499b6fb8f-fnjd9\" (UID: \"1ae29235-cca2-4a37-80ae-65fef194d3fc\") " pod="openshift-console/downloads-5499b6fb8f-fnjd9"
Jul 02 13:47:13.875678 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.875644    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-fnjd9"
Jul 02 13:47:13.934525 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.934497    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf"]
Jul 02 13:47:13.935476 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.935453    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmg9w\" (UniqueName: \"kubernetes.io/projected/585ed721-7d68-462d-9431-17135ff5475b-kube-api-access-nmg9w\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.935564 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.935548    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/585ed721-7d68-462d-9431-17135ff5475b-registry-tls\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.935605 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.935585    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/585ed721-7d68-462d-9431-17135ff5475b-registry-certificates\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.935662 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.935616    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/585ed721-7d68-462d-9431-17135ff5475b-trusted-ca\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.935716 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.935671    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/585ed721-7d68-462d-9431-17135ff5475b-image-registry-private-configuration\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.935761 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.935714    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/585ed721-7d68-462d-9431-17135ff5475b-ca-trust-extracted\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.935761 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.935740    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/585ed721-7d68-462d-9431-17135ff5475b-bound-sa-token\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.935857 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.935778    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/585ed721-7d68-462d-9431-17135ff5475b-installation-pull-secrets\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.936497 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.936267    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/585ed721-7d68-462d-9431-17135ff5475b-ca-trust-extracted\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.936638 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.936538    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/585ed721-7d68-462d-9431-17135ff5475b-registry-certificates\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.937810 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.937739    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/585ed721-7d68-462d-9431-17135ff5475b-trusted-ca\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.938538 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.938465    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/585ed721-7d68-462d-9431-17135ff5475b-registry-tls\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.938653 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.938592    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/585ed721-7d68-462d-9431-17135ff5475b-image-registry-private-configuration\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.938803 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.938775    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/585ed721-7d68-462d-9431-17135ff5475b-installation-pull-secrets\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.939036 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:47:13.939013    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd16a0359_1dc8_426e_bcdb_6f4143196fcd.slice/crio-c10f11c6e92f4088f5e86a8135a2bef7f619a53bf185d2a9602603f8caac93e4 WatchSource:0}: Error finding container c10f11c6e92f4088f5e86a8135a2bef7f619a53bf185d2a9602603f8caac93e4: Status 404 returned error can't find the container with id c10f11c6e92f4088f5e86a8135a2bef7f619a53bf185d2a9602603f8caac93e4
Jul 02 13:47:13.945571 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.945551    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmg9w\" (UniqueName: \"kubernetes.io/projected/585ed721-7d68-462d-9431-17135ff5475b-kube-api-access-nmg9w\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:13.954782 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:13.954709    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/585ed721-7d68-462d-9431-17135ff5475b-bound-sa-token\") pod \"image-registry-644d865c6b-mk8jh\" (UID: \"585ed721-7d68-462d-9431-17135ff5475b\") " pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:14.012211 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.012173    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:14.016942 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.016905    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-fnjd9"]
Jul 02 13:47:14.020319 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:47:14.020295    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae29235_cca2_4a37_80ae_65fef194d3fc.slice/crio-e74b00eb47efedf92994468bd301aac154fa3c0c2bc1ae759eb92d971d3e576a WatchSource:0}: Error finding container e74b00eb47efedf92994468bd301aac154fa3c0c2bc1ae759eb92d971d3e576a: Status 404 returned error can't find the container with id e74b00eb47efedf92994468bd301aac154fa3c0c2bc1ae759eb92d971d3e576a
Jul 02 13:47:14.140237 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.140202    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:47:14.142695 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.142671    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:47:14.150048 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.150023    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-644d865c6b-mk8jh"]
Jul 02 13:47:14.153300 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:47:14.153277    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod585ed721_7d68_462d_9431_17135ff5475b.slice/crio-9776e1fe8a3badeb1b73e487203f5ee96c0b5eae1eb5be0847179bfd25802e9c WatchSource:0}: Error finding container 9776e1fe8a3badeb1b73e487203f5ee96c0b5eae1eb5be0847179bfd25802e9c: Status 404 returned error can't find the container with id 9776e1fe8a3badeb1b73e487203f5ee96c0b5eae1eb5be0847179bfd25802e9c
Jul 02 13:47:14.355301 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.355262    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" event={"ID":"585ed721-7d68-462d-9431-17135ff5475b","Type":"ContainerStarted","Data":"d5d1d56a0d4ce719b218b03012b31e8af87b25b4126de7d40703c1e73008d87e"}
Jul 02 13:47:14.355473 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.355314    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" event={"ID":"585ed721-7d68-462d-9431-17135ff5475b","Type":"ContainerStarted","Data":"9776e1fe8a3badeb1b73e487203f5ee96c0b5eae1eb5be0847179bfd25802e9c"}
Jul 02 13:47:14.355473 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.355413    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:14.356418 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.356390    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-fnjd9" event={"ID":"1ae29235-cca2-4a37-80ae-65fef194d3fc","Type":"ContainerStarted","Data":"e74b00eb47efedf92994468bd301aac154fa3c0c2bc1ae759eb92d971d3e576a"}
Jul 02 13:47:14.357615 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.357590    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf" event={"ID":"d16a0359-1dc8-426e-bcdb-6f4143196fcd","Type":"ContainerStarted","Data":"c10f11c6e92f4088f5e86a8135a2bef7f619a53bf185d2a9602603f8caac93e4"}
Jul 02 13:47:14.357763 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.357752    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:47:14.359194 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.359175    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6c987749f-vpfgn"
Jul 02 13:47:14.379417 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:14.379371    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" podStartSLOduration=1.379354181 podStartE2EDuration="1.379354181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:14.378064618 +0000 UTC m=+119.883189124" watchObservedRunningTime="2026-07-02 13:47:14.379354181 +0000 UTC m=+119.884478685"
Jul 02 13:47:15.753834 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:15.753746    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:15.878425 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:15.878379    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:16.034325 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:16.034243    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:16.510439 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:16.510398    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:17.063633 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:17.063593    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:47:17.066263 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:17.066233    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d753af15-bfcf-4f37-8d5e-d894521e6e5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxgcc\" (UID: \"d753af15-bfcf-4f37-8d5e-d894521e6e5b\") " pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:47:17.354172 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:17.354091    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmzxj\""
Jul 02 13:47:17.362311 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:17.362270    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nxgcc"
Jul 02 13:47:17.371616 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:17.371578    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf" event={"ID":"d16a0359-1dc8-426e-bcdb-6f4143196fcd","Type":"ContainerStarted","Data":"88cb2bed776a9f183540d20dc205874681058a9771119cfcf943a995ee5907c8"}
Jul 02 13:47:17.506372 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:17.506310    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84d699996d-p4cvf" podStartSLOduration=1.6855239210000001 podStartE2EDuration="4.50628876s" totalImagesPullingTime="2.820764839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:17.402703838 +0000 UTC m=+122.907828344" watchObservedRunningTime="2026-07-02 13:47:17.50628876 +0000 UTC m=+123.011413263"
Jul 02 13:47:17.506910 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:17.506889    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nxgcc"]
Jul 02 13:47:17.510208 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:47:17.510177    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd753af15_bfcf_4f37_8d5e_d894521e6e5b.slice/crio-b8e6348a7c146b1909f21e486fb51c5aecf77d0ca0d788e44a52e4d0075e3c0d WatchSource:0}: Error finding container b8e6348a7c146b1909f21e486fb51c5aecf77d0ca0d788e44a52e4d0075e3c0d: Status 404 returned error can't find the container with id b8e6348a7c146b1909f21e486fb51c5aecf77d0ca0d788e44a52e4d0075e3c0d
Jul 02 13:47:18.380166 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:18.380125    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxgcc" event={"ID":"d753af15-bfcf-4f37-8d5e-d894521e6e5b","Type":"ContainerStarted","Data":"b8e6348a7c146b1909f21e486fb51c5aecf77d0ca0d788e44a52e4d0075e3c0d"}
Jul 02 13:47:20.107878 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:20.107837    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:20.177247 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:20.177214    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:22.399092 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:22.399054    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxgcc" event={"ID":"d753af15-bfcf-4f37-8d5e-d894521e6e5b","Type":"ContainerStarted","Data":"ab5d3b8c3c6e3dcd61b76165f311c634d0c1a7bf73fbaecb688f0032e9a52fab"}
Jul 02 13:47:23.407098 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:23.407056    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxgcc" event={"ID":"d753af15-bfcf-4f37-8d5e-d894521e6e5b","Type":"ContainerStarted","Data":"5a2207627e0ebcc6022cd074ae5b3d5be7241dce48cac2e792f69cd6d9cbbcf1"}
Jul 02 13:47:23.719571 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:23.719455    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:47:23.723192 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:23.723163    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d01be1dd-2bb1-409d-aaaa-6094d6e05f26-metrics-certs\") pod \"network-metrics-daemon-whrk8\" (UID: \"d01be1dd-2bb1-409d-aaaa-6094d6e05f26\") " pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:47:23.792074 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:23.792038    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:23.913725 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:23.913687    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4vzgw\""
Jul 02 13:47:23.921148 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:23.921108    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-whrk8"
Jul 02 13:47:23.931214 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:23.931174    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:24.049116 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.049044    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:24.096779 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.096731    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"]
Jul 02 13:47:24.101599 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.101215    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.104821 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.104269    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 02 13:47:24.104821 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.104402    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 02 13:47:24.104821 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.104593    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-6j7wq\""
Jul 02 13:47:24.105088 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.104979    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 02 13:47:24.105847 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.105366    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 02 13:47:24.105847 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.105576    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 02 13:47:24.113497 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.113392    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-whrk8"]
Jul 02 13:47:24.114914 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.114851    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"]
Jul 02 13:47:24.118584 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:47:24.118559    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd01be1dd_2bb1_409d_aaaa_6094d6e05f26.slice/crio-02864debbfe1f590ddf6957e2259479d5a1c24b0c3e48108064cd05da384de6d WatchSource:0}: Error finding container 02864debbfe1f590ddf6957e2259479d5a1c24b0c3e48108064cd05da384de6d: Status 404 returned error can't find the container with id 02864debbfe1f590ddf6957e2259479d5a1c24b0c3e48108064cd05da384de6d
Jul 02 13:47:24.131780 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.131750    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:24.223330 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.223285    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.223544 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.223340    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.223623 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.223562    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.223623 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.223602    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6qjf\" (UniqueName: \"kubernetes.io/projected/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-kube-api-access-j6qjf\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.324075 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.324036    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.324263 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.324085    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qjf\" (UniqueName: \"kubernetes.io/projected/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-kube-api-access-j6qjf\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.324263 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.324119    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.324263 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.324149    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.324433 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:47:24.324298    2567 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 02 13:47:24.324433 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:47:24.324357    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-tls podName:0c4da376-ce5b-4cfd-a22d-aacd9920f2fe nodeName:}" failed. No retries permitted until 2026-07-02 13:47:24.824339336 +0000 UTC m=+130.329463824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-tls") pod "prometheus-operator-7b7d896497-dtx5r" (UID: "0c4da376-ce5b-4cfd-a22d-aacd9920f2fe") : secret "prometheus-operator-tls" not found
Jul 02 13:47:24.325352 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.325313    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.328459 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.328427    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.337634 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.337601    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6qjf\" (UniqueName: \"kubernetes.io/projected/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-kube-api-access-j6qjf\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.414110 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.414025    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-whrk8" event={"ID":"d01be1dd-2bb1-409d-aaaa-6094d6e05f26","Type":"ContainerStarted","Data":"02864debbfe1f590ddf6957e2259479d5a1c24b0c3e48108064cd05da384de6d"}
Jul 02 13:47:24.829940 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.829901    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:24.830124 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:47:24.830089    2567 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 02 13:47:24.830197 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:47:24.830180    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-tls podName:0c4da376-ce5b-4cfd-a22d-aacd9920f2fe nodeName:}" failed. No retries permitted until 2026-07-02 13:47:25.830160928 +0000 UTC m=+131.335285413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-tls") pod "prometheus-operator-7b7d896497-dtx5r" (UID: "0c4da376-ce5b-4cfd-a22d-aacd9920f2fe") : secret "prometheus-operator-tls" not found
Jul 02 13:47:24.954107 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:24.954061    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:25.755141 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.755109    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54d767775f-x2hsb"]
Jul 02 13:47:25.758288 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.758264    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:25.761061 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.761034    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 02 13:47:25.761423 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.761399    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 02 13:47:25.761808 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.761743    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 02 13:47:25.762364 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.762060    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 02 13:47:25.762364 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.762250    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hlnlb\""
Jul 02 13:47:25.762364 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.762287    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 02 13:47:25.764352 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.764260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:25.769811 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.768832    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 02 13:47:25.770515 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.770460    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54d767775f-x2hsb"]
Jul 02 13:47:25.838939 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.838903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:25.841568 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.841538    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c4da376-ce5b-4cfd-a22d-aacd9920f2fe-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-dtx5r\" (UID: \"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:25.863703 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.863664    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:25.913448 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.913415    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"
Jul 02 13:47:25.939780 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.939745    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-console-config\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:25.939943 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.939858    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-oauth-config\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:25.939943 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.939878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-serving-cert\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:25.939943 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.939902    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-trusted-ca-bundle\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:25.940057 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.939946    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rjrf\" (UniqueName: \"kubernetes.io/projected/60e09b75-7c78-42b8-a654-57a72c36f468-kube-api-access-5rjrf\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:25.940057 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.939980    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-service-ca\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:25.940057 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:25.939997    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-oauth-serving-cert\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.041359 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.041268    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-oauth-config\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.041359 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.041322    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-serving-cert\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.041359 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.041344    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-trusted-ca-bundle\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.041674 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.041372    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rjrf\" (UniqueName: \"kubernetes.io/projected/60e09b75-7c78-42b8-a654-57a72c36f468-kube-api-access-5rjrf\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.041674 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.041401    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-service-ca\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.041674 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.041424    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-oauth-serving-cert\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.041674 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.041453    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-console-config\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.042406 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.042253    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-console-config\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.042406 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.042302    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-service-ca\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.042406 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.042378    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-trusted-ca-bundle\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.042645 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.042457    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-oauth-serving-cert\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.044050 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.044030    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-oauth-config\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.044196 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.044173    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-serving-cert\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.049644 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.049620    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rjrf\" (UniqueName: \"kubernetes.io/projected/60e09b75-7c78-42b8-a654-57a72c36f468-kube-api-access-5rjrf\") pod \"console-54d767775f-x2hsb\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") " pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:26.070546 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:26.070507    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:32.226045 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:32.225993    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:32.293432 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:32.293397    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:33.063588 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.063554    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54d767775f-x2hsb"]
Jul 02 13:47:33.067254 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:47:33.067222    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e09b75_7c78_42b8_a654_57a72c36f468.slice/crio-e7bd1512ea1823bbe3d5b3aeb9bfbfa3052d75efe67029fb5bbb0e75a6dbd205 WatchSource:0}: Error finding container e7bd1512ea1823bbe3d5b3aeb9bfbfa3052d75efe67029fb5bbb0e75a6dbd205: Status 404 returned error can't find the container with id e7bd1512ea1823bbe3d5b3aeb9bfbfa3052d75efe67029fb5bbb0e75a6dbd205
Jul 02 13:47:33.083672 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.083638    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-dtx5r"]
Jul 02 13:47:33.087107 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:47:33.087063    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4da376_ce5b_4cfd_a22d_aacd9920f2fe.slice/crio-30c2aaec2850f7320301a54c87c18ecbbf154a19f7244a603bc67149c53c78e8 WatchSource:0}: Error finding container 30c2aaec2850f7320301a54c87c18ecbbf154a19f7244a603bc67149c53c78e8: Status 404 returned error can't find the container with id 30c2aaec2850f7320301a54c87c18ecbbf154a19f7244a603bc67149c53c78e8
Jul 02 13:47:33.450283 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.450195    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxgcc" event={"ID":"d753af15-bfcf-4f37-8d5e-d894521e6e5b","Type":"ContainerStarted","Data":"ca6a30ce3add7f3b81c98adfd68d29530e1485061fcc5ab26c6c7bd9e4fe26bf"}
Jul 02 13:47:33.451879 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.451847    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r" event={"ID":"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe","Type":"ContainerStarted","Data":"30c2aaec2850f7320301a54c87c18ecbbf154a19f7244a603bc67149c53c78e8"}
Jul 02 13:47:33.453573 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.453543    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-fnjd9" event={"ID":"1ae29235-cca2-4a37-80ae-65fef194d3fc","Type":"ContainerStarted","Data":"ef89cfd69940fbbd4d17499c024eb764b56fc811b638961732f11b9415999f65"}
Jul 02 13:47:33.453767 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.453745    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5499b6fb8f-fnjd9"
Jul 02 13:47:33.455848 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.455820    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-whrk8" event={"ID":"d01be1dd-2bb1-409d-aaaa-6094d6e05f26","Type":"ContainerStarted","Data":"bb5be83926b4d1d1727ef3ece26b246789ac0920fdca98b05f34cf7bf058f6bf"}
Jul 02 13:47:33.455972 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.455871    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-whrk8" event={"ID":"d01be1dd-2bb1-409d-aaaa-6094d6e05f26","Type":"ContainerStarted","Data":"1c09a89a80b606fe0e35894a7be7860722d6b3ae44cb1e838bdf7d58f65e7b9b"}
Jul 02 13:47:33.457165 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.457129    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d767775f-x2hsb" event={"ID":"60e09b75-7c78-42b8-a654-57a72c36f468","Type":"ContainerStarted","Data":"e7bd1512ea1823bbe3d5b3aeb9bfbfa3052d75efe67029fb5bbb0e75a6dbd205"}
Jul 02 13:47:33.465555 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.465527    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5499b6fb8f-fnjd9"
Jul 02 13:47:33.472268 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.472005    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nxgcc" podStartSLOduration=33.232191831 podStartE2EDuration="48.471980685s" totalImagesPullingTime="15.239788854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:46:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:33.471643169 +0000 UTC m=+138.976767698" watchObservedRunningTime="2026-07-02 13:47:33.471980685 +0000 UTC m=+138.977105189"
Jul 02 13:47:33.490361 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.490305    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5499b6fb8f-fnjd9" podStartSLOduration=1.545010142 podStartE2EDuration="20.490281139s" totalImagesPullingTime="18.945270997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:33.489036869 +0000 UTC m=+138.994161409" watchObservedRunningTime="2026-07-02 13:47:33.490281139 +0000 UTC m=+138.995405643"
Jul 02 13:47:33.509055 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.508992    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-whrk8" podStartSLOduration=129.725854722 podStartE2EDuration="2m18.508969333s" totalImagesPullingTime="8.783114611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:33.507661841 +0000 UTC m=+139.012786360" watchObservedRunningTime="2026-07-02 13:47:33.508969333 +0000 UTC m=+139.014093839"
Jul 02 13:47:33.604600 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.604562    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:33.671559 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:33.671521    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:34.019216 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:34.018882    2567 patch_prober.go:28] interesting pod/image-registry-644d865c6b-mk8jh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 13:47:34.019216 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:34.018953    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" podUID="585ed721-7d68-462d-9431-17135ff5475b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:47:35.367719 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:35.367674    2567 patch_prober.go:28] interesting pod/image-registry-644d865c6b-mk8jh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 13:47:35.368196 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:35.367741    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" podUID="585ed721-7d68-462d-9431-17135ff5475b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:47:35.481116 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:35.481029    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r" event={"ID":"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe","Type":"ContainerStarted","Data":"7c047d344df19e9715517f51d920219e47a66356d012ab6803b6843df24f16ea"}
Jul 02 13:47:35.595217 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:35.595169    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:37.493037 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:37.492991    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r" event={"ID":"0c4da376-ce5b-4cfd-a22d-aacd9920f2fe","Type":"ContainerStarted","Data":"027b59792268c54f611bdb6ce4db5a6c7fe060b31b958ab7e3d5bb6b2845e6a8"}
Jul 02 13:47:37.494873 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:37.494842    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d767775f-x2hsb" event={"ID":"60e09b75-7c78-42b8-a654-57a72c36f468","Type":"ContainerStarted","Data":"78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527"}
Jul 02 13:47:37.516378 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:37.516326    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-dtx5r" podStartSLOduration=11.411897391 podStartE2EDuration="13.516309893s" totalImagesPullingTime="2.104412502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:47:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:37.514625744 +0000 UTC m=+143.019750248" watchObservedRunningTime="2026-07-02 13:47:37.516309893 +0000 UTC m=+143.021434397"
Jul 02 13:47:37.536649 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:37.536586    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54d767775f-x2hsb" podStartSLOduration=8.658702201 podStartE2EDuration="12.53656179s" totalImagesPullingTime="3.877859589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:37.535454462 +0000 UTC m=+143.040578998" watchObservedRunningTime="2026-07-02 13:47:37.53656179 +0000 UTC m=+143.041686299"
Jul 02 13:47:39.214072 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.214030    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:39.425979 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.425946    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:39.522008 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.521916    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-zwkbz"]
Jul 02 13:47:39.568841 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.568808    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vdkwc"]
Jul 02 13:47:39.569129 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.569103    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.573741 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.572343    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 02 13:47:39.573741 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.573001    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 02 13:47:39.573741 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.573267    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-926nd\""
Jul 02 13:47:39.574992 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.574966    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 02 13:47:39.584200 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.582692    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-zwkbz"]
Jul 02 13:47:39.584200 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.582848    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.588520 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.588497    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mczhp\""
Jul 02 13:47:39.589727 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.588872    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 02 13:47:39.590064 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.588994    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 02 13:47:39.590362 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.589063    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 02 13:47:39.599559 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.599396    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:39.662803 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.662764    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-textfile\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.662803 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.662816    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40e2287b-c0f8-49ef-b444-da316078e626-sys\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.663064 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.662839    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/219d776d-ee12-4b78-a891-e26ffd6d8b41-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.663064 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.662864    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.663064 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.662941    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.663064 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.663027    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.663064 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.663064    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40e2287b-c0f8-49ef-b444-da316078e626-root\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.663320 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.663090    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40e2287b-c0f8-49ef-b444-da316078e626-metrics-client-ca\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.663320 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.663110    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hltr5\" (UniqueName: \"kubernetes.io/projected/40e2287b-c0f8-49ef-b444-da316078e626-kube-api-access-hltr5\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.663320 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.663144    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/219d776d-ee12-4b78-a891-e26ffd6d8b41-metrics-client-ca\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.663320 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.663192    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-accelerators-collector-config\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.663320 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.663273    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-wtmp\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.663320 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.663307    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.663642 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.663341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwt58\" (UniqueName: \"kubernetes.io/projected/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-api-access-rwt58\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.663642 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.663366    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-tls\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.706765 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.706726    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:39.764202 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764153    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.764202 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764208    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40e2287b-c0f8-49ef-b444-da316078e626-root\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.764582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764228    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40e2287b-c0f8-49ef-b444-da316078e626-metrics-client-ca\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.764582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764251    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hltr5\" (UniqueName: \"kubernetes.io/projected/40e2287b-c0f8-49ef-b444-da316078e626-kube-api-access-hltr5\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.764582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/219d776d-ee12-4b78-a891-e26ffd6d8b41-metrics-client-ca\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.764582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764335    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-accelerators-collector-config\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.764582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764388    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-wtmp\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.764582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764412    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.764582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764444    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwt58\" (UniqueName: \"kubernetes.io/projected/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-api-access-rwt58\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.764582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764470    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-tls\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.764582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764530    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-textfile\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.764582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764559    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40e2287b-c0f8-49ef-b444-da316078e626-sys\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.764582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764580    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/219d776d-ee12-4b78-a891-e26ffd6d8b41-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.765070 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764604    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.765070 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.764638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.765070 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.765042    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-wtmp\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.765499 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.765448    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.766217 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.766191    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40e2287b-c0f8-49ef-b444-da316078e626-root\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.766325 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:47:39.766290    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 02 13:47:39.766385 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:47:39.766358    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-tls podName:40e2287b-c0f8-49ef-b444-da316078e626 nodeName:}" failed. No retries permitted until 2026-07-02 13:47:40.266336476 +0000 UTC m=+145.771460960 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-tls") pod "node-exporter-vdkwc" (UID: "40e2287b-c0f8-49ef-b444-da316078e626") : secret "node-exporter-tls" not found
Jul 02 13:47:39.766563 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.766539    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/219d776d-ee12-4b78-a891-e26ffd6d8b41-metrics-client-ca\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.766725 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.766194    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-textfile\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.766879 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.766861    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40e2287b-c0f8-49ef-b444-da316078e626-sys\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.766976 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.766881    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/219d776d-ee12-4b78-a891-e26ffd6d8b41-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.767070 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.766941    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40e2287b-c0f8-49ef-b444-da316078e626-metrics-client-ca\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.767160 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.767137    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-accelerators-collector-config\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.768821 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.768151    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.768821 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.768647    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.769521 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.769455    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.778126 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.777991    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hltr5\" (UniqueName: \"kubernetes.io/projected/40e2287b-c0f8-49ef-b444-da316078e626-kube-api-access-hltr5\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:39.778126 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.778043    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwt58\" (UniqueName: \"kubernetes.io/projected/219d776d-ee12-4b78-a891-e26ffd6d8b41-kube-api-access-rwt58\") pod \"kube-state-metrics-845b954455-zwkbz\" (UID: \"219d776d-ee12-4b78-a891-e26ffd6d8b41\") " pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.854402 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.854365    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:39.896456 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.896408    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz"
Jul 02 13:47:39.955687 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:39.955657    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:40.092930 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:40.092897    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:40.099771 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:40.099740    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-zwkbz"]
Jul 02 13:47:40.102423 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:47:40.102391    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod219d776d_ee12_4b78_a891_e26ffd6d8b41.slice/crio-94d5be11563f6d723a26df67bf1a40ef60e47d8252d3d9dc202174954c395aa5 WatchSource:0}: Error finding container 94d5be11563f6d723a26df67bf1a40ef60e47d8252d3d9dc202174954c395aa5: Status 404 returned error can't find the container with id 94d5be11563f6d723a26df67bf1a40ef60e47d8252d3d9dc202174954c395aa5
Jul 02 13:47:40.270312 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:40.270270    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-tls\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:40.273425 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:40.273393    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40e2287b-c0f8-49ef-b444-da316078e626-node-exporter-tls\") pod \"node-exporter-vdkwc\" (UID: \"40e2287b-c0f8-49ef-b444-da316078e626\") " pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:40.453768 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:40.453726    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:40.506991 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:40.506956    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vdkwc"
Jul 02 13:47:40.509258 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:40.509171    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz" event={"ID":"219d776d-ee12-4b78-a891-e26ffd6d8b41","Type":"ContainerStarted","Data":"94d5be11563f6d723a26df67bf1a40ef60e47d8252d3d9dc202174954c395aa5"}
Jul 02 13:47:40.538093 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:40.538062    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:40.589300 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:47:40.589263    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e2287b_c0f8_49ef_b444_da316078e626.slice/crio-5b2560ae033a94042c67736551dd86144aaf92b4c816f612a161ed8a3b8b6a47 WatchSource:0}: Error finding container 5b2560ae033a94042c67736551dd86144aaf92b4c816f612a161ed8a3b8b6a47: Status 404 returned error can't find the container with id 5b2560ae033a94042c67736551dd86144aaf92b4c816f612a161ed8a3b8b6a47
Jul 02 13:47:40.679984 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:40.679946    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:41.092884 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:41.092843    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:41.515465 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:41.515371    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vdkwc" event={"ID":"40e2287b-c0f8-49ef-b444-da316078e626","Type":"ContainerStarted","Data":"5b2560ae033a94042c67736551dd86144aaf92b4c816f612a161ed8a3b8b6a47"}
Jul 02 13:47:41.825118 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:41.825082    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:42.457401 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:42.457371    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:42.560775 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:42.560735    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:42.988407 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:42.988356    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:43.486367 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:43.486331    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:43.525379 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:43.525341    2567 generic.go:356] "Generic (PLEG): container finished" podID="40e2287b-c0f8-49ef-b444-da316078e626" containerID="84ec0a8696dc0507981b3d9a853cec9c0803ff9e6cd070eb42997848679162ad" exitCode=0
Jul 02 13:47:43.525577 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:43.525423    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vdkwc" event={"ID":"40e2287b-c0f8-49ef-b444-da316078e626","Type":"ContainerDied","Data":"84ec0a8696dc0507981b3d9a853cec9c0803ff9e6cd070eb42997848679162ad"}
Jul 02 13:47:43.527811 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:43.527781    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz" event={"ID":"219d776d-ee12-4b78-a891-e26ffd6d8b41","Type":"ContainerStarted","Data":"14a4df1da901d7178de0c78a36d0b65d811bd3371ff134d8526494388ea26912"}
Jul 02 13:47:43.527930 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:43.527840    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz" event={"ID":"219d776d-ee12-4b78-a891-e26ffd6d8b41","Type":"ContainerStarted","Data":"88aabac1e02518a013b81f84df79258a6246b01b6cd9444986148a5584acfb58"}
Jul 02 13:47:43.527930 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:43.527871    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz" event={"ID":"219d776d-ee12-4b78-a891-e26ffd6d8b41","Type":"ContainerStarted","Data":"f5e15025960184b1a93a247589e181b6cf640eda8342790fde78fa9bf148b0e7"}
Jul 02 13:47:43.568623 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:43.568568    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-zwkbz" podStartSLOduration=2.234985748 podStartE2EDuration="4.568548656s" totalImagesPullingTime="2.333562908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:43.567352812 +0000 UTC m=+149.072477329" watchObservedRunningTime="2026-07-02 13:47:43.568548656 +0000 UTC m=+149.073673160"
Jul 02 13:47:44.016945 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:44.016908    2567 patch_prober.go:28] interesting pod/image-registry-644d865c6b-mk8jh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 13:47:44.017104 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:44.016964    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" podUID="585ed721-7d68-462d-9431-17135ff5475b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:47:44.183107 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:44.183055    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:44.325120 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:44.325073    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:44.426938 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:44.426907    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:44.495668 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:44.495561    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:44.534091 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:44.534045    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vdkwc" event={"ID":"40e2287b-c0f8-49ef-b444-da316078e626","Type":"ContainerStarted","Data":"0cb1ac7a413ad5d66507c01ed9778e902284243800fa30739fd2d8604d392ddb"}
Jul 02 13:47:44.534269 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:44.534142    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vdkwc" event={"ID":"40e2287b-c0f8-49ef-b444-da316078e626","Type":"ContainerStarted","Data":"1bab06271d5420101df39081770adf20e7a69ef6f8dac4d0ef7378a1a6281cc0"}
Jul 02 13:47:44.554524 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:44.554448    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vdkwc" podStartSLOduration=3.5875578729999997 podStartE2EDuration="5.554430539s" totalImagesPullingTime="1.848722284s" totalInitContainerRuntime="118.150382ms" isStatefulPod=true podCreationTimestamp="2026-07-02 13:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:44.553617228 +0000 UTC m=+150.058741759" watchObservedRunningTime="2026-07-02 13:47:44.554430539 +0000 UTC m=+150.059555144"
Jul 02 13:47:45.366531 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:45.366497    2567 patch_prober.go:28] interesting pod/image-registry-644d865c6b-mk8jh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 13:47:45.366895 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:45.366557    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" podUID="585ed721-7d68-462d-9431-17135ff5475b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:47:45.775408 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:45.775305    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:46.071554 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:46.071507    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:46.071739 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:46.071577    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:47:46.073015 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:46.072994    2567 patch_prober.go:28] interesting pod/console-54d767775f-x2hsb container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.17:8443/health\": dial tcp 10.134.0.17:8443: connect: connection refused" start-of-body=
Jul 02 13:47:46.073075 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:46.073042    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-54d767775f-x2hsb" podUID="60e09b75-7c78-42b8-a654-57a72c36f468" containerName="console" probeResult="failure" output="Get \"https://10.134.0.17:8443/health\": dial tcp 10.134.0.17:8443: connect: connection refused"
Jul 02 13:47:51.339444 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:51.339401    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:52.339649 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:47:52.339601    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9h5lq" podUID="b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6"
Jul 02 13:47:52.566753 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:52.566721    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:47:54.016564 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:54.016533    2567 patch_prober.go:28] interesting pod/image-registry-644d865c6b-mk8jh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 13:47:54.017036 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:54.016590    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" podUID="585ed721-7d68-462d-9431-17135ff5475b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:47:54.017036 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:54.016636    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:47:54.017224 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:54.017190    2567 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"d5d1d56a0d4ce719b218b03012b31e8af87b25b4126de7d40703c1e73008d87e"} pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 02 13:47:54.020761 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:54.020737    2567 patch_prober.go:28] interesting pod/image-registry-644d865c6b-mk8jh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 13:47:54.020879 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:54.020773    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" podUID="585ed721-7d68-462d-9431-17135ff5475b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:47:54.874508 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:54.874449    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54d767775f-x2hsb"]
Jul 02 13:47:55.008037 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:55.008003    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:55.105544 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:55.105511    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:55.164360 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:55.164266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:55.762036 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:55.762004    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:57.215918 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:57.215878    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:47:57.218138 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:57.218119    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6-metrics-tls\") pod \"dns-default-9h5lq\" (UID: \"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6\") " pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:47:57.370440 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:57.370407    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pnc7w\""
Jul 02 13:47:57.377920 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:57.377888    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:47:57.572301 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:57.572125    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9h5lq"]
Jul 02 13:47:57.574834 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:47:57.574794    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb253b67b_e1fc_4f4b_a20d_25cd3d07c7f6.slice/crio-9d2be302b00ffe100d868c7a0851e38e63a0719b1b2c7d3967377ba5d20a0c95 WatchSource:0}: Error finding container 9d2be302b00ffe100d868c7a0851e38e63a0719b1b2c7d3967377ba5d20a0c95: Status 404 returned error can't find the container with id 9d2be302b00ffe100d868c7a0851e38e63a0719b1b2c7d3967377ba5d20a0c95
Jul 02 13:47:57.586537 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:57.586511    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9h5lq" event={"ID":"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6","Type":"ContainerStarted","Data":"9d2be302b00ffe100d868c7a0851e38e63a0719b1b2c7d3967377ba5d20a0c95"}
Jul 02 13:47:57.690916 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:57.690884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:57.757753 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:57.757676    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:58.031089 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:58.031011    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:58.132141 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:58.132108    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:59.855085 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:59.855052    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:47:59.956603 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:47:59.956570    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:00.520359 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:00.520325    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:00.600372 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:00.600338    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9h5lq" event={"ID":"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6","Type":"ContainerStarted","Data":"92d0a4b8dd571e430cf9fa0a385cfbe2ca2596c5d6d103b3c96a2af3c3555b49"}
Jul 02 13:48:00.600551 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:00.600388    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9h5lq" event={"ID":"b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6","Type":"ContainerStarted","Data":"bbc1888fe9c5917f50afabd7fa23915c43b7a266b74bc44752e6b85971a52b58"}
Jul 02 13:48:00.600551 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:00.600465    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:48:00.625569 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:00.625516    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9h5lq" podStartSLOduration=129.592229494 podStartE2EDuration="2m11.625477185s" totalImagesPullingTime="2.033247691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:48:00.623245722 +0000 UTC m=+166.128370225" watchObservedRunningTime="2026-07-02 13:48:00.625477185 +0000 UTC m=+166.130601725"
Jul 02 13:48:00.627299 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:00.627271    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:00.814726 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:00.814699    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:02.238126 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:02.238082    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:02.327747 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:02.327713    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:04.021079 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:04.021045    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:48:04.173243 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:04.173209    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:04.273048 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:04.272952    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:04.924341 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:04.924310    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:04.978867 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:04.978834    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:10.606194 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:10.606121    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9h5lq"
Jul 02 13:48:11.379230 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:11.379198    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:11.487151 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:11.487113    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:14.651918 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:14.651888    2567 generic.go:356] "Generic (PLEG): container finished" podID="75095516-19ab-4843-aeb8-8b4df9fc1005" containerID="207099eb6e6133785da15fc35dfffed158ac3c7bafa3f37420862a6558d3c330" exitCode=0
Jul 02 13:48:14.652312 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:14.651935    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-z4sj4" event={"ID":"75095516-19ab-4843-aeb8-8b4df9fc1005","Type":"ContainerDied","Data":"207099eb6e6133785da15fc35dfffed158ac3c7bafa3f37420862a6558d3c330"}
Jul 02 13:48:14.652312 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:14.652263    2567 scope.go:117] "RemoveContainer" containerID="207099eb6e6133785da15fc35dfffed158ac3c7bafa3f37420862a6558d3c330"
Jul 02 13:48:15.640195 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:15.640160    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9h5lq_b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6/dns/0.log"
Jul 02 13:48:15.647175 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:15.647147    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9h5lq_b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6/kube-rbac-proxy/0.log"
Jul 02 13:48:15.656949 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:15.656920    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-z4sj4" event={"ID":"75095516-19ab-4843-aeb8-8b4df9fc1005","Type":"ContainerStarted","Data":"f4cdd6776b009615c0bba352c2a1d526d9979c56886c9c18cfda96736f8f37a2"}
Jul 02 13:48:16.194711 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:16.194681    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zqz2f_747984cc-ab52-45dc-99ef-53c93e5bd101/dns-node-resolver/0.log"
Jul 02 13:48:17.936563 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:17.936527    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:18.060962 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:18.060932    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:19.036773 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:19.036733    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" podUID="585ed721-7d68-462d-9431-17135ff5475b" containerName="registry" containerID="cri-o://d5d1d56a0d4ce719b218b03012b31e8af87b25b4126de7d40703c1e73008d87e" gracePeriod=30
Jul 02 13:48:19.894639 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:19.894596    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-54d767775f-x2hsb" podUID="60e09b75-7c78-42b8-a654-57a72c36f468" containerName="console" containerID="cri-o://78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527" gracePeriod=15
Jul 02 13:48:19.940622 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:19.940587    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:20.029816 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.029781    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:20.218797 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.218766    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:48:20.304418 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.304382    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-console-config\") pod \"60e09b75-7c78-42b8-a654-57a72c36f468\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") "
Jul 02 13:48:20.304622 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.304432    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rjrf\" (UniqueName: \"kubernetes.io/projected/60e09b75-7c78-42b8-a654-57a72c36f468-kube-api-access-5rjrf\") pod \"60e09b75-7c78-42b8-a654-57a72c36f468\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") "
Jul 02 13:48:20.304622 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.304462    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-oauth-config\") pod \"60e09b75-7c78-42b8-a654-57a72c36f468\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") "
Jul 02 13:48:20.304622 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.304512    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-serving-cert\") pod \"60e09b75-7c78-42b8-a654-57a72c36f468\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") "
Jul 02 13:48:20.304622 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.304540    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-oauth-serving-cert\") pod \"60e09b75-7c78-42b8-a654-57a72c36f468\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") "
Jul 02 13:48:20.304622 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.304557    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-service-ca\") pod \"60e09b75-7c78-42b8-a654-57a72c36f468\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") "
Jul 02 13:48:20.304622 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.304597    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-trusted-ca-bundle\") pod \"60e09b75-7c78-42b8-a654-57a72c36f468\" (UID: \"60e09b75-7c78-42b8-a654-57a72c36f468\") "
Jul 02 13:48:20.305052 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.304991    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-console-config" (OuterVolumeSpecName: "console-config") pod "60e09b75-7c78-42b8-a654-57a72c36f468" (UID: "60e09b75-7c78-42b8-a654-57a72c36f468"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:48:20.305052 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.305020    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-service-ca" (OuterVolumeSpecName: "service-ca") pod "60e09b75-7c78-42b8-a654-57a72c36f468" (UID: "60e09b75-7c78-42b8-a654-57a72c36f468"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:48:20.305202 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.305054    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "60e09b75-7c78-42b8-a654-57a72c36f468" (UID: "60e09b75-7c78-42b8-a654-57a72c36f468"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:48:20.305202 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.305145    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "60e09b75-7c78-42b8-a654-57a72c36f468" (UID: "60e09b75-7c78-42b8-a654-57a72c36f468"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:48:20.306802 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.306782    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e09b75-7c78-42b8-a654-57a72c36f468-kube-api-access-5rjrf" (OuterVolumeSpecName: "kube-api-access-5rjrf") pod "60e09b75-7c78-42b8-a654-57a72c36f468" (UID: "60e09b75-7c78-42b8-a654-57a72c36f468"). InnerVolumeSpecName "kube-api-access-5rjrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:48:20.306802 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.306792    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "60e09b75-7c78-42b8-a654-57a72c36f468" (UID: "60e09b75-7c78-42b8-a654-57a72c36f468"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:48:20.306914 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.306801    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "60e09b75-7c78-42b8-a654-57a72c36f468" (UID: "60e09b75-7c78-42b8-a654-57a72c36f468"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:48:20.405540 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.405457    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rjrf\" (UniqueName: \"kubernetes.io/projected/60e09b75-7c78-42b8-a654-57a72c36f468-kube-api-access-5rjrf\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:48:20.405540 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.405506    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-oauth-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:48:20.405540 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.405515    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60e09b75-7c78-42b8-a654-57a72c36f468-console-serving-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:48:20.405540 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.405524    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-oauth-serving-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:48:20.405540 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.405533    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-service-ca\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:48:20.405540 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.405541    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-trusted-ca-bundle\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:48:20.405869 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.405549    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60e09b75-7c78-42b8-a654-57a72c36f468-console-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:48:20.677306 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.677271    2567 generic.go:356] "Generic (PLEG): container finished" podID="60e09b75-7c78-42b8-a654-57a72c36f468" containerID="78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527" exitCode=0
Jul 02 13:48:20.677431 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.677320    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54d767775f-x2hsb"
Jul 02 13:48:20.677431 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.677364    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d767775f-x2hsb" event={"ID":"60e09b75-7c78-42b8-a654-57a72c36f468","Type":"ContainerDied","Data":"78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527"}
Jul 02 13:48:20.677431 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.677410    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54d767775f-x2hsb" event={"ID":"60e09b75-7c78-42b8-a654-57a72c36f468","Type":"ContainerDied","Data":"e7bd1512ea1823bbe3d5b3aeb9bfbfa3052d75efe67029fb5bbb0e75a6dbd205"}
Jul 02 13:48:20.677431 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.677431    2567 scope.go:117] "RemoveContainer" containerID="78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527"
Jul 02 13:48:20.679214 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.679194    2567 generic.go:356] "Generic (PLEG): container finished" podID="585ed721-7d68-462d-9431-17135ff5475b" containerID="d5d1d56a0d4ce719b218b03012b31e8af87b25b4126de7d40703c1e73008d87e" exitCode=0
Jul 02 13:48:20.679324 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.679266    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" event={"ID":"585ed721-7d68-462d-9431-17135ff5475b","Type":"ContainerDied","Data":"d5d1d56a0d4ce719b218b03012b31e8af87b25b4126de7d40703c1e73008d87e"}
Jul 02 13:48:20.689754 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.689729    2567 scope.go:117] "RemoveContainer" containerID="78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527"
Jul 02 13:48:20.690137 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:48:20.690050    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527\": container with ID starting with 78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527 not found: ID does not exist" containerID="78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527"
Jul 02 13:48:20.690137 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.690077    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527"} err="failed to get container status \"78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527\": rpc error: code = NotFound desc = could not find container \"78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527\": container with ID starting with 78849ab4194e8e2ac2a0875108f480b68e97a063d365cc803feafc9df5584527 not found: ID does not exist"
Jul 02 13:48:20.702749 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.702722    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54d767775f-x2hsb"]
Jul 02 13:48:20.705949 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.705924    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54d767775f-x2hsb"]
Jul 02 13:48:20.903597 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:20.903568    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60e09b75-7c78-42b8-a654-57a72c36f468" path="/var/lib/kubelet/pods/60e09b75-7c78-42b8-a654-57a72c36f468/volumes"
Jul 02 13:48:21.685292 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:21.685255    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh" event={"ID":"585ed721-7d68-462d-9431-17135ff5475b","Type":"ContainerStarted","Data":"01f51f8b662865c53673802c888865e435cee5194a2d463230519f8d5da276ff"}
Jul 02 13:48:21.685693 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:21.685391    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:48:21.996757 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:21.996683    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:22.080110 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:22.080077    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:26.934263 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:26.934230    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:26.996458 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:26.996420    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:34.917766 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:34.917731    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:35.092259 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:35.092227    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:42.692943 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:42.692918    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-644d865c6b-mk8jh"
Jul 02 13:48:42.984299 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:42.984187    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:43.085094 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:43.085049    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:52.780389 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:52.780356    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:58.798919 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:58.798883    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:59.000687 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:48:59.000656    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:00.152547 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:00.152508    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:00.260490 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:00.260463    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:04.922333 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:04.922292    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:04.987145 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:04.987114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:07.919557 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.919522    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b4678d466-rg5mg"]
Jul 02 13:49:07.919942 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.919821    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e09b75-7c78-42b8-a654-57a72c36f468" containerName="console"
Jul 02 13:49:07.919942 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.919835    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e09b75-7c78-42b8-a654-57a72c36f468" containerName="console"
Jul 02 13:49:07.919942 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.919906    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e09b75-7c78-42b8-a654-57a72c36f468" containerName="console"
Jul 02 13:49:07.922057 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.922034    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:07.924797 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.924759    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 02 13:49:07.925761 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.925738    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 02 13:49:07.926243 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.925911    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 02 13:49:07.926243 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.926021    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 02 13:49:07.926243 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.926165    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 02 13:49:07.927240 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.927221    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hlnlb\""
Jul 02 13:49:07.934500 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.934458    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4678d466-rg5mg"]
Jul 02 13:49:07.936183 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.936159    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 02 13:49:07.981534 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:07.981466    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:08.081197 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.081159    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:08.086382 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.086347    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbjn8\" (UniqueName: \"kubernetes.io/projected/307847f1-f751-441a-8cf7-7af3151c4d63-kube-api-access-tbjn8\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.086550 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.086397    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-console-config\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.086550 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.086443    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-serving-cert\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.086550 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.086497    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-service-ca\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.086550 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.086524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-oauth-config\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.086550 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.086548    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-oauth-serving-cert\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.086749 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.086601    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-trusted-ca-bundle\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.181337 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.181241    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:08.187551 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.187521    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-service-ca\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.187789 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.187769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-oauth-config\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.187900 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.187885    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-oauth-serving-cert\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.188018 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.188004    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-trusted-ca-bundle\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.188177 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.188160    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbjn8\" (UniqueName: \"kubernetes.io/projected/307847f1-f751-441a-8cf7-7af3151c4d63-kube-api-access-tbjn8\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.188312 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.188299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-console-config\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.188640 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.188439    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-service-ca\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.188934 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.188911    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-oauth-serving-cert\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.189121 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.189090    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-console-config\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.189121 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.189110    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-trusted-ca-bundle\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.189285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.189114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-serving-cert\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.190593 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.190566    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-oauth-config\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.191387 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.191354    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-serving-cert\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.196625 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.196598    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbjn8\" (UniqueName: \"kubernetes.io/projected/307847f1-f751-441a-8cf7-7af3151c4d63-kube-api-access-tbjn8\") pod \"console-5b4678d466-rg5mg\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") " pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.243588 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.243545    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:08.281821 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.281778    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:08.402554 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.402234    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4678d466-rg5mg"]
Jul 02 13:49:08.405780 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:49:08.405747    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod307847f1_f751_441a_8cf7_7af3151c4d63.slice/crio-497be2c9f7b2a8624ff204339d3f25507f4c6efd1f8bed3c3d39ed9af3ef20da WatchSource:0}: Error finding container 497be2c9f7b2a8624ff204339d3f25507f4c6efd1f8bed3c3d39ed9af3ef20da: Status 404 returned error can't find the container with id 497be2c9f7b2a8624ff204339d3f25507f4c6efd1f8bed3c3d39ed9af3ef20da
Jul 02 13:49:08.720446 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.720357    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:08.779816 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.779786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:08.858365 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.858332    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4678d466-rg5mg" event={"ID":"307847f1-f751-441a-8cf7-7af3151c4d63","Type":"ContainerStarted","Data":"f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565"}
Jul 02 13:49:08.858531 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.858386    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4678d466-rg5mg" event={"ID":"307847f1-f751-441a-8cf7-7af3151c4d63","Type":"ContainerStarted","Data":"497be2c9f7b2a8624ff204339d3f25507f4c6efd1f8bed3c3d39ed9af3ef20da"}
Jul 02 13:49:08.878860 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:08.878811    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b4678d466-rg5mg" podStartSLOduration=1.878795476 podStartE2EDuration="1.878795476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:49:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:49:08.877813672 +0000 UTC m=+234.382938195" watchObservedRunningTime="2026-07-02 13:49:08.878795476 +0000 UTC m=+234.383919980"
Jul 02 13:49:18.244651 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:18.244602    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:18.245135 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:18.244721    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:18.249649 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:18.249626    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:18.896936 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:18.896909    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:49:19.117971 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:19.117934    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:19.219031 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:19.218948    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:19.299349 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:19.299313    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:45.787638 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:45.787562    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:45.890664 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:45.890631    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:46.118599 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:49:46.118567    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:07.768376 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:07.768344    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:14.099020 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:14.098989    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:14.236849 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:14.236816    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:14.839247 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:14.839222    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 13:50:14.841615 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:14.841586    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 13:50:14.846551 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:14.846530    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 13:50:14.848980 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:14.848961    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 13:50:14.850680 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:14.850662    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 02 13:50:17.931476 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:17.931426    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:18.070794 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:18.070752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:18.128684 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:18.128639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:18.666693 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:18.666657    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:18.766630 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:18.766596    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:26.094298 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:26.094244    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:26.156426 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:26.156394    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:28.487357 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:28.487320    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b4678d466-rg5mg"]
Jul 02 13:50:28.599332 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:28.599300    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:28.699677 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:28.699650    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:28.758123 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:28.758040    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:34.995964 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:34.995929    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:35.104139 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:35.104112    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:53.510002 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.509940    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5b4678d466-rg5mg" podUID="307847f1-f751-441a-8cf7-7af3151c4d63" containerName="console" containerID="cri-o://f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565" gracePeriod=15
Jul 02 13:50:53.749966 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.749937    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:50:53.811707 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.811666    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbjn8\" (UniqueName: \"kubernetes.io/projected/307847f1-f751-441a-8cf7-7af3151c4d63-kube-api-access-tbjn8\") pod \"307847f1-f751-441a-8cf7-7af3151c4d63\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") "
Jul 02 13:50:53.811887 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.811771    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-oauth-serving-cert\") pod \"307847f1-f751-441a-8cf7-7af3151c4d63\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") "
Jul 02 13:50:53.811887 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.811797    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-serving-cert\") pod \"307847f1-f751-441a-8cf7-7af3151c4d63\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") "
Jul 02 13:50:53.811887 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.811820    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-oauth-config\") pod \"307847f1-f751-441a-8cf7-7af3151c4d63\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") "
Jul 02 13:50:53.811887 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.811846    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-trusted-ca-bundle\") pod \"307847f1-f751-441a-8cf7-7af3151c4d63\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") "
Jul 02 13:50:53.811887 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.811873    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-console-config\") pod \"307847f1-f751-441a-8cf7-7af3151c4d63\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") "
Jul 02 13:50:53.812135 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.811898    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-service-ca\") pod \"307847f1-f751-441a-8cf7-7af3151c4d63\" (UID: \"307847f1-f751-441a-8cf7-7af3151c4d63\") "
Jul 02 13:50:53.812326 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.812301    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "307847f1-f751-441a-8cf7-7af3151c4d63" (UID: "307847f1-f751-441a-8cf7-7af3151c4d63"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:50:53.812429 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.812316    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "307847f1-f751-441a-8cf7-7af3151c4d63" (UID: "307847f1-f751-441a-8cf7-7af3151c4d63"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:50:53.812429 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.812325    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-console-config" (OuterVolumeSpecName: "console-config") pod "307847f1-f751-441a-8cf7-7af3151c4d63" (UID: "307847f1-f751-441a-8cf7-7af3151c4d63"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:50:53.812543 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.812423    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-service-ca" (OuterVolumeSpecName: "service-ca") pod "307847f1-f751-441a-8cf7-7af3151c4d63" (UID: "307847f1-f751-441a-8cf7-7af3151c4d63"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:50:53.813998 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.813978    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "307847f1-f751-441a-8cf7-7af3151c4d63" (UID: "307847f1-f751-441a-8cf7-7af3151c4d63"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:50:53.814409 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.814382    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "307847f1-f751-441a-8cf7-7af3151c4d63" (UID: "307847f1-f751-441a-8cf7-7af3151c4d63"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:50:53.814515 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.814411    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/307847f1-f751-441a-8cf7-7af3151c4d63-kube-api-access-tbjn8" (OuterVolumeSpecName: "kube-api-access-tbjn8") pod "307847f1-f751-441a-8cf7-7af3151c4d63" (UID: "307847f1-f751-441a-8cf7-7af3151c4d63"). InnerVolumeSpecName "kube-api-access-tbjn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:50:53.912443 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.912403    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-oauth-serving-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:50:53.912443 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.912430    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-serving-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:50:53.912443 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.912439    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/307847f1-f751-441a-8cf7-7af3151c4d63-console-oauth-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:50:53.912443 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.912448    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-trusted-ca-bundle\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:50:53.912443 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.912457    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-console-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:50:53.912749 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.912466    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/307847f1-f751-441a-8cf7-7af3151c4d63-service-ca\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:50:53.912749 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:53.912475    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbjn8\" (UniqueName: \"kubernetes.io/projected/307847f1-f751-441a-8cf7-7af3151c4d63-kube-api-access-tbjn8\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:50:54.242191 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:54.242103    2567 generic.go:356] "Generic (PLEG): container finished" podID="307847f1-f751-441a-8cf7-7af3151c4d63" containerID="f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565" exitCode=0
Jul 02 13:50:54.242191 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:54.242152    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4678d466-rg5mg"
Jul 02 13:50:54.242388 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:54.242196    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4678d466-rg5mg" event={"ID":"307847f1-f751-441a-8cf7-7af3151c4d63","Type":"ContainerDied","Data":"f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565"}
Jul 02 13:50:54.242388 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:54.242244    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4678d466-rg5mg" event={"ID":"307847f1-f751-441a-8cf7-7af3151c4d63","Type":"ContainerDied","Data":"497be2c9f7b2a8624ff204339d3f25507f4c6efd1f8bed3c3d39ed9af3ef20da"}
Jul 02 13:50:54.242388 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:54.242265    2567 scope.go:117] "RemoveContainer" containerID="f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565"
Jul 02 13:50:54.250604 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:54.250585    2567 scope.go:117] "RemoveContainer" containerID="f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565"
Jul 02 13:50:54.250871 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:50:54.250853    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565\": container with ID starting with f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565 not found: ID does not exist" containerID="f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565"
Jul 02 13:50:54.250918 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:54.250881    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565"} err="failed to get container status \"f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565\": rpc error: code = NotFound desc = could not find container \"f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565\": container with ID starting with f909f69b1e11ed412385ad7f4eb7fa21fc4e6ea193bc9f9c25f188c7f60c0565 not found: ID does not exist"
Jul 02 13:50:54.268723 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:54.268693    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b4678d466-rg5mg"]
Jul 02 13:50:54.274458 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:54.274437    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b4678d466-rg5mg"]
Jul 02 13:50:54.902609 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:50:54.902580    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="307847f1-f751-441a-8cf7-7af3151c4d63" path="/var/lib/kubelet/pods/307847f1-f751-441a-8cf7-7af3151c4d63/volumes"
Jul 02 13:52:13.178503 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:13.178447    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:14.406315 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.406285    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"]
Jul 02 13:52:14.406723 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.406588    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="307847f1-f751-441a-8cf7-7af3151c4d63" containerName="console"
Jul 02 13:52:14.406723 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.406599    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="307847f1-f751-441a-8cf7-7af3151c4d63" containerName="console"
Jul 02 13:52:14.406723 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.406642    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="307847f1-f751-441a-8cf7-7af3151c4d63" containerName="console"
Jul 02 13:52:14.409552 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.409536    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:14.412151 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.412124    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 02 13:52:14.412275 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.412124    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 02 13:52:14.413075 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.413056    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tw4nb\""
Jul 02 13:52:14.418944 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.418919    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"]
Jul 02 13:52:14.422750 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.422728    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frds6\" (UniqueName: \"kubernetes.io/projected/b97ed97f-ba3b-4dda-be6b-83ca70425851-kube-api-access-frds6\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:14.422865 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.422789    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:14.422865 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.422850    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:14.523536 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.523473    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:14.523734 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.523554    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:14.523734 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.523621    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frds6\" (UniqueName: \"kubernetes.io/projected/b97ed97f-ba3b-4dda-be6b-83ca70425851-kube-api-access-frds6\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:14.523893 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.523874    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:14.524006 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.523984    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:14.532072 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.532036    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frds6\" (UniqueName: \"kubernetes.io/projected/b97ed97f-ba3b-4dda-be6b-83ca70425851-kube-api-access-frds6\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:14.719159 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.719068    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:14.845950 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.845922    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:14.853620 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.853567    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"]
Jul 02 13:52:14.856030 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:52:14.856005    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb97ed97f_ba3b_4dda_be6b_83ca70425851.slice/crio-cfc0f3af74f5c73fc09cde304b9cb260ccf707b0f51eed2671914143fff05c2d WatchSource:0}: Error finding container cfc0f3af74f5c73fc09cde304b9cb260ccf707b0f51eed2671914143fff05c2d: Status 404 returned error can't find the container with id cfc0f3af74f5c73fc09cde304b9cb260ccf707b0f51eed2671914143fff05c2d
Jul 02 13:52:14.857946 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:14.857930    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 13:52:15.541857 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:15.541820    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v" event={"ID":"b97ed97f-ba3b-4dda-be6b-83ca70425851","Type":"ContainerStarted","Data":"cfc0f3af74f5c73fc09cde304b9cb260ccf707b0f51eed2671914143fff05c2d"}
Jul 02 13:52:22.572830 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:22.572796    2567 generic.go:356] "Generic (PLEG): container finished" podID="b97ed97f-ba3b-4dda-be6b-83ca70425851" containerID="70c70acb86aa4edef2cedb0bdb94a215206b370c5666d645002fa32f25597616" exitCode=0
Jul 02 13:52:22.573207 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:22.572847    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v" event={"ID":"b97ed97f-ba3b-4dda-be6b-83ca70425851","Type":"ContainerDied","Data":"70c70acb86aa4edef2cedb0bdb94a215206b370c5666d645002fa32f25597616"}
Jul 02 13:52:25.586285 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:25.586247    2567 generic.go:356] "Generic (PLEG): container finished" podID="b97ed97f-ba3b-4dda-be6b-83ca70425851" containerID="d9e0accd968139e87b4bdb10fc8391f03f34a36da007633189ef3da318abe25d" exitCode=0
Jul 02 13:52:25.586717 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:25.586300    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v" event={"ID":"b97ed97f-ba3b-4dda-be6b-83ca70425851","Type":"ContainerDied","Data":"d9e0accd968139e87b4bdb10fc8391f03f34a36da007633189ef3da318abe25d"}
Jul 02 13:52:33.622108 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:33.622073    2567 generic.go:356] "Generic (PLEG): container finished" podID="b97ed97f-ba3b-4dda-be6b-83ca70425851" containerID="2847cec96295b8083a9f56988e00cfd3e4ccc6b2a5c730fa8e72cff9417c1812" exitCode=0
Jul 02 13:52:33.622528 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:33.622166    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v" event={"ID":"b97ed97f-ba3b-4dda-be6b-83ca70425851","Type":"ContainerDied","Data":"2847cec96295b8083a9f56988e00cfd3e4ccc6b2a5c730fa8e72cff9417c1812"}
Jul 02 13:52:34.753967 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:34.753942    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:34.903172 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:34.903086    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-bundle\") pod \"b97ed97f-ba3b-4dda-be6b-83ca70425851\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") "
Jul 02 13:52:34.903172 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:34.903150    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-util\") pod \"b97ed97f-ba3b-4dda-be6b-83ca70425851\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") "
Jul 02 13:52:34.903393 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:34.903209    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frds6\" (UniqueName: \"kubernetes.io/projected/b97ed97f-ba3b-4dda-be6b-83ca70425851-kube-api-access-frds6\") pod \"b97ed97f-ba3b-4dda-be6b-83ca70425851\" (UID: \"b97ed97f-ba3b-4dda-be6b-83ca70425851\") "
Jul 02 13:52:34.903730 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:34.903704    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-bundle" (OuterVolumeSpecName: "bundle") pod "b97ed97f-ba3b-4dda-be6b-83ca70425851" (UID: "b97ed97f-ba3b-4dda-be6b-83ca70425851"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 13:52:34.905349 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:34.905329    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b97ed97f-ba3b-4dda-be6b-83ca70425851-kube-api-access-frds6" (OuterVolumeSpecName: "kube-api-access-frds6") pod "b97ed97f-ba3b-4dda-be6b-83ca70425851" (UID: "b97ed97f-ba3b-4dda-be6b-83ca70425851"). InnerVolumeSpecName "kube-api-access-frds6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:52:34.910668 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:34.910643    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-util" (OuterVolumeSpecName: "util") pod "b97ed97f-ba3b-4dda-be6b-83ca70425851" (UID: "b97ed97f-ba3b-4dda-be6b-83ca70425851"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 13:52:35.004276 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:35.004233    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-util\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:52:35.004276 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:35.004263    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frds6\" (UniqueName: \"kubernetes.io/projected/b97ed97f-ba3b-4dda-be6b-83ca70425851-kube-api-access-frds6\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:52:35.004276 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:35.004274    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b97ed97f-ba3b-4dda-be6b-83ca70425851-bundle\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:52:35.630970 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:35.630936    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v" event={"ID":"b97ed97f-ba3b-4dda-be6b-83ca70425851","Type":"ContainerDied","Data":"cfc0f3af74f5c73fc09cde304b9cb260ccf707b0f51eed2671914143fff05c2d"}
Jul 02 13:52:35.631134 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:35.630984    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfc0f3af74f5c73fc09cde304b9cb260ccf707b0f51eed2671914143fff05c2d"
Jul 02 13:52:35.631134 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:35.630947    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589cft7v"
Jul 02 13:52:40.574533 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:40.574438    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:40.669444 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:40.669410    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:46.795731 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:46.795701    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:47.115395 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:47.115362    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:47.362013 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:47.361979    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:47.651028 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:47.650998    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:47.923447 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:47.923345    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:48.008727 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:48.008682    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:48.077692 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:48.077658    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:48.279985 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:48.279904    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:48.541099 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:48.541009    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:48.642769 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:48.642743    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:48.750368 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:48.750336    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:52:55.813431 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:52:55.813398    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:53:58.766698 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.766660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:53:58.844545 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.844512    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7c747fbc6-sxqt7"]
Jul 02 13:53:58.844790 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.844778    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b97ed97f-ba3b-4dda-be6b-83ca70425851" containerName="pull"
Jul 02 13:53:58.844857 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.844791    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b97ed97f-ba3b-4dda-be6b-83ca70425851" containerName="pull"
Jul 02 13:53:58.844857 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.844800    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b97ed97f-ba3b-4dda-be6b-83ca70425851" containerName="extract"
Jul 02 13:53:58.844857 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.844806    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b97ed97f-ba3b-4dda-be6b-83ca70425851" containerName="extract"
Jul 02 13:53:58.844857 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.844819    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b97ed97f-ba3b-4dda-be6b-83ca70425851" containerName="util"
Jul 02 13:53:58.844857 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.844825    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b97ed97f-ba3b-4dda-be6b-83ca70425851" containerName="util"
Jul 02 13:53:58.845093 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.844863    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b97ed97f-ba3b-4dda-be6b-83ca70425851" containerName="extract"
Jul 02 13:53:58.851627 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.851324    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:53:58.854047 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.854020    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-p9npl"]
Jul 02 13:53:58.854975 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.854956    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 02 13:53:58.855076 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.855003    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 02 13:53:58.855076 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.855019    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-7cvws\""
Jul 02 13:53:58.855185 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.855002    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 02 13:53:58.856219 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.856203    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl"
Jul 02 13:53:58.859327 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.859304    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7c747fbc6-sxqt7"]
Jul 02 13:53:58.860552 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.860528    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-nf82v\""
Jul 02 13:53:58.860801 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.860783    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 02 13:53:58.862232 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.862209    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:53:58.869913 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.869779    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-p9npl"]
Jul 02 13:53:58.941460 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.941435    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:53:58.974148 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.974116    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e189a31-5139-4291-bc2d-90f85ad58384-cert\") pod \"llmisvc-controller-manager-67d95fdc48-p9npl\" (UID: \"5e189a31-5139-4291-bc2d-90f85ad58384\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl"
Jul 02 13:53:58.974283 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.974165    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj5cv\" (UniqueName: \"kubernetes.io/projected/bc1826ae-bd90-47bb-aa97-62a8439972d7-kube-api-access-cj5cv\") pod \"kserve-controller-manager-7c747fbc6-sxqt7\" (UID: \"bc1826ae-bd90-47bb-aa97-62a8439972d7\") " pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:53:58.974283 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.974198    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc1826ae-bd90-47bb-aa97-62a8439972d7-cert\") pod \"kserve-controller-manager-7c747fbc6-sxqt7\" (UID: \"bc1826ae-bd90-47bb-aa97-62a8439972d7\") " pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:53:58.974283 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:58.974233    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wmcd\" (UniqueName: \"kubernetes.io/projected/5e189a31-5139-4291-bc2d-90f85ad58384-kube-api-access-6wmcd\") pod \"llmisvc-controller-manager-67d95fdc48-p9npl\" (UID: \"5e189a31-5139-4291-bc2d-90f85ad58384\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl"
Jul 02 13:53:59.043575 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.043477    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:53:59.074815 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.074776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj5cv\" (UniqueName: \"kubernetes.io/projected/bc1826ae-bd90-47bb-aa97-62a8439972d7-kube-api-access-cj5cv\") pod \"kserve-controller-manager-7c747fbc6-sxqt7\" (UID: \"bc1826ae-bd90-47bb-aa97-62a8439972d7\") " pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:53:59.074815 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.074816    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc1826ae-bd90-47bb-aa97-62a8439972d7-cert\") pod \"kserve-controller-manager-7c747fbc6-sxqt7\" (UID: \"bc1826ae-bd90-47bb-aa97-62a8439972d7\") " pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:53:59.075062 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.074846    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wmcd\" (UniqueName: \"kubernetes.io/projected/5e189a31-5139-4291-bc2d-90f85ad58384-kube-api-access-6wmcd\") pod \"llmisvc-controller-manager-67d95fdc48-p9npl\" (UID: \"5e189a31-5139-4291-bc2d-90f85ad58384\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl"
Jul 02 13:53:59.075062 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.074889    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e189a31-5139-4291-bc2d-90f85ad58384-cert\") pod \"llmisvc-controller-manager-67d95fdc48-p9npl\" (UID: \"5e189a31-5139-4291-bc2d-90f85ad58384\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl"
Jul 02 13:53:59.075062 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:53:59.074966    2567 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 02 13:53:59.075062 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:53:59.075019    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e189a31-5139-4291-bc2d-90f85ad58384-cert podName:5e189a31-5139-4291-bc2d-90f85ad58384 nodeName:}" failed. No retries permitted until 2026-07-02 13:53:59.575003359 +0000 UTC m=+525.080127842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5e189a31-5139-4291-bc2d-90f85ad58384-cert") pod "llmisvc-controller-manager-67d95fdc48-p9npl" (UID: "5e189a31-5139-4291-bc2d-90f85ad58384") : secret "llmisvc-webhook-server-cert" not found
Jul 02 13:53:59.077374 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.077351    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc1826ae-bd90-47bb-aa97-62a8439972d7-cert\") pod \"kserve-controller-manager-7c747fbc6-sxqt7\" (UID: \"bc1826ae-bd90-47bb-aa97-62a8439972d7\") " pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:53:59.086898 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.086868    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj5cv\" (UniqueName: \"kubernetes.io/projected/bc1826ae-bd90-47bb-aa97-62a8439972d7-kube-api-access-cj5cv\") pod \"kserve-controller-manager-7c747fbc6-sxqt7\" (UID: \"bc1826ae-bd90-47bb-aa97-62a8439972d7\") " pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:53:59.087180 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.087159    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wmcd\" (UniqueName: \"kubernetes.io/projected/5e189a31-5139-4291-bc2d-90f85ad58384-kube-api-access-6wmcd\") pod \"llmisvc-controller-manager-67d95fdc48-p9npl\" (UID: \"5e189a31-5139-4291-bc2d-90f85ad58384\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl"
Jul 02 13:53:59.166380 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.166343    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:53:59.199125 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.199093    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:53:59.312392 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.312363    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7c747fbc6-sxqt7"]
Jul 02 13:53:59.315879 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:53:59.315855    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc1826ae_bd90_47bb_aa97_62a8439972d7.slice/crio-493a697b6d415440cafdcd2b67da216f797c859ea6295ea4992eeac8988d2b6a WatchSource:0}: Error finding container 493a697b6d415440cafdcd2b67da216f797c859ea6295ea4992eeac8988d2b6a: Status 404 returned error can't find the container with id 493a697b6d415440cafdcd2b67da216f797c859ea6295ea4992eeac8988d2b6a
Jul 02 13:53:59.580016 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.579865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e189a31-5139-4291-bc2d-90f85ad58384-cert\") pod \"llmisvc-controller-manager-67d95fdc48-p9npl\" (UID: \"5e189a31-5139-4291-bc2d-90f85ad58384\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl"
Jul 02 13:53:59.582197 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.582152    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e189a31-5139-4291-bc2d-90f85ad58384-cert\") pod \"llmisvc-controller-manager-67d95fdc48-p9npl\" (UID: \"5e189a31-5139-4291-bc2d-90f85ad58384\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl"
Jul 02 13:53:59.773637 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.773606    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl"
Jul 02 13:53:59.871058 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.870957    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:53:59.913869 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.913834    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-p9npl"]
Jul 02 13:53:59.917868 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:53:59.917833    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5e189a31_5139_4291_bc2d_90f85ad58384.slice/crio-0fea8859727e96b4418095616fc076b61c864ab98181b84a9e58823f420249ae WatchSource:0}: Error finding container 0fea8859727e96b4418095616fc076b61c864ab98181b84a9e58823f420249ae: Status 404 returned error can't find the container with id 0fea8859727e96b4418095616fc076b61c864ab98181b84a9e58823f420249ae
Jul 02 13:53:59.940390 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.940353    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl" event={"ID":"5e189a31-5139-4291-bc2d-90f85ad58384","Type":"ContainerStarted","Data":"0fea8859727e96b4418095616fc076b61c864ab98181b84a9e58823f420249ae"}
Jul 02 13:53:59.941998 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.941965    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7" event={"ID":"bc1826ae-bd90-47bb-aa97-62a8439972d7","Type":"ContainerStarted","Data":"493a697b6d415440cafdcd2b67da216f797c859ea6295ea4992eeac8988d2b6a"}
Jul 02 13:53:59.969877 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:53:59.969769    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:54:00.034251 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:00.034217    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:54:02.957790 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:02.957740    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl" event={"ID":"5e189a31-5139-4291-bc2d-90f85ad58384","Type":"ContainerStarted","Data":"50f1952e118e037bd8a03e59342fab562c52949e818d385f7c7ab9fcb5d8efdd"}
Jul 02 13:54:02.958253 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:02.957944    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl"
Jul 02 13:54:02.959357 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:02.959327    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7" event={"ID":"bc1826ae-bd90-47bb-aa97-62a8439972d7","Type":"ContainerStarted","Data":"8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380"}
Jul 02 13:54:02.959468 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:02.959427    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:54:02.979117 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:02.979070    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl" podStartSLOduration=2.090345768 podStartE2EDuration="4.97905435s" totalImagesPullingTime="2.888708582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:53:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:54:02.977939209 +0000 UTC m=+528.483063739" watchObservedRunningTime="2026-07-02 13:54:02.97905435 +0000 UTC m=+528.484178854"
Jul 02 13:54:02.996165 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:02.996114    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7" podStartSLOduration=2.038655253 podStartE2EDuration="4.996096988s" totalImagesPullingTime="2.957441735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:53:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:54:02.99530645 +0000 UTC m=+528.500430969" watchObservedRunningTime="2026-07-02 13:54:02.996096988 +0000 UTC m=+528.501221492"
Jul 02 13:54:04.414701 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:04.414669    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:54:20.574890 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:20.574811    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:54:33.964651 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:33.964620    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-p9npl"
Jul 02 13:54:33.967985 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:33.967963    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:54:35.522534 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.522499    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7c747fbc6-sxqt7"]
Jul 02 13:54:35.522969 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.522674    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7" podUID="bc1826ae-bd90-47bb-aa97-62a8439972d7" containerName="manager" containerID="cri-o://8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380" gracePeriod=10
Jul 02 13:54:35.547777 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.547751    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7c747fbc6-hrc9l"]
Jul 02 13:54:35.549790 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.549774    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l"
Jul 02 13:54:35.562854 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.562829    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7c747fbc6-hrc9l"]
Jul 02 13:54:35.645694 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.645668    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqtjz\" (UniqueName: \"kubernetes.io/projected/69470978-4c8f-455c-9f12-154c6a6f64fd-kube-api-access-sqtjz\") pod \"kserve-controller-manager-7c747fbc6-hrc9l\" (UID: \"69470978-4c8f-455c-9f12-154c6a6f64fd\") " pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l"
Jul 02 13:54:35.645797 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.645720    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69470978-4c8f-455c-9f12-154c6a6f64fd-cert\") pod \"kserve-controller-manager-7c747fbc6-hrc9l\" (UID: \"69470978-4c8f-455c-9f12-154c6a6f64fd\") " pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l"
Jul 02 13:54:35.746433 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.746410    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqtjz\" (UniqueName: \"kubernetes.io/projected/69470978-4c8f-455c-9f12-154c6a6f64fd-kube-api-access-sqtjz\") pod \"kserve-controller-manager-7c747fbc6-hrc9l\" (UID: \"69470978-4c8f-455c-9f12-154c6a6f64fd\") " pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l"
Jul 02 13:54:35.746551 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.746503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69470978-4c8f-455c-9f12-154c6a6f64fd-cert\") pod \"kserve-controller-manager-7c747fbc6-hrc9l\" (UID: \"69470978-4c8f-455c-9f12-154c6a6f64fd\") " pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l"
Jul 02 13:54:35.748780 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.748756    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69470978-4c8f-455c-9f12-154c6a6f64fd-cert\") pod \"kserve-controller-manager-7c747fbc6-hrc9l\" (UID: \"69470978-4c8f-455c-9f12-154c6a6f64fd\") " pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l"
Jul 02 13:54:35.754880 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.754856    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqtjz\" (UniqueName: \"kubernetes.io/projected/69470978-4c8f-455c-9f12-154c6a6f64fd-kube-api-access-sqtjz\") pod \"kserve-controller-manager-7c747fbc6-hrc9l\" (UID: \"69470978-4c8f-455c-9f12-154c6a6f64fd\") " pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l"
Jul 02 13:54:35.768976 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.768950    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:54:35.847338 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.847302    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc1826ae-bd90-47bb-aa97-62a8439972d7-cert\") pod \"bc1826ae-bd90-47bb-aa97-62a8439972d7\" (UID: \"bc1826ae-bd90-47bb-aa97-62a8439972d7\") "
Jul 02 13:54:35.847556 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.847376    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cj5cv\" (UniqueName: \"kubernetes.io/projected/bc1826ae-bd90-47bb-aa97-62a8439972d7-kube-api-access-cj5cv\") pod \"bc1826ae-bd90-47bb-aa97-62a8439972d7\" (UID: \"bc1826ae-bd90-47bb-aa97-62a8439972d7\") "
Jul 02 13:54:35.849521 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.849468    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1826ae-bd90-47bb-aa97-62a8439972d7-kube-api-access-cj5cv" (OuterVolumeSpecName: "kube-api-access-cj5cv") pod "bc1826ae-bd90-47bb-aa97-62a8439972d7" (UID: "bc1826ae-bd90-47bb-aa97-62a8439972d7"). InnerVolumeSpecName "kube-api-access-cj5cv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:54:35.849642 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.849526    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc1826ae-bd90-47bb-aa97-62a8439972d7-cert" (OuterVolumeSpecName: "cert") pod "bc1826ae-bd90-47bb-aa97-62a8439972d7" (UID: "bc1826ae-bd90-47bb-aa97-62a8439972d7"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:54:35.910152 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.910099    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l"
Jul 02 13:54:35.948938 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.948894    2567 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc1826ae-bd90-47bb-aa97-62a8439972d7-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:54:35.948938 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:35.948933    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj5cv\" (UniqueName: \"kubernetes.io/projected/bc1826ae-bd90-47bb-aa97-62a8439972d7-kube-api-access-cj5cv\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:54:36.041630 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.041597    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7c747fbc6-hrc9l"]
Jul 02 13:54:36.044604 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:54:36.044580    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69470978_4c8f_455c_9f12_154c6a6f64fd.slice/crio-a9f0b35517514ffc0c6c02d933c7cea55f16cf677859c553f4358e44b3dfe6e4 WatchSource:0}: Error finding container a9f0b35517514ffc0c6c02d933c7cea55f16cf677859c553f4358e44b3dfe6e4: Status 404 returned error can't find the container with id a9f0b35517514ffc0c6c02d933c7cea55f16cf677859c553f4358e44b3dfe6e4
Jul 02 13:54:36.083113 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.083073    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l" event={"ID":"69470978-4c8f-455c-9f12-154c6a6f64fd","Type":"ContainerStarted","Data":"a9f0b35517514ffc0c6c02d933c7cea55f16cf677859c553f4358e44b3dfe6e4"}
Jul 02 13:54:36.084211 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.084187    2567 generic.go:356] "Generic (PLEG): container finished" podID="bc1826ae-bd90-47bb-aa97-62a8439972d7" containerID="8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380" exitCode=0
Jul 02 13:54:36.084338 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.084219    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7" event={"ID":"bc1826ae-bd90-47bb-aa97-62a8439972d7","Type":"ContainerDied","Data":"8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380"}
Jul 02 13:54:36.084338 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.084247    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7" event={"ID":"bc1826ae-bd90-47bb-aa97-62a8439972d7","Type":"ContainerDied","Data":"493a697b6d415440cafdcd2b67da216f797c859ea6295ea4992eeac8988d2b6a"}
Jul 02 13:54:36.084338 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.084244    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7c747fbc6-sxqt7"
Jul 02 13:54:36.084338 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.084267    2567 scope.go:117] "RemoveContainer" containerID="8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380"
Jul 02 13:54:36.092180 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.092160    2567 scope.go:117] "RemoveContainer" containerID="8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380"
Jul 02 13:54:36.092448 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:54:36.092428    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380\": container with ID starting with 8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380 not found: ID does not exist" containerID="8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380"
Jul 02 13:54:36.092538 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.092469    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380"} err="failed to get container status \"8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380\": rpc error: code = NotFound desc = could not find container \"8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380\": container with ID starting with 8d2496c543cd547d3f8369af4ba57704bdfeefbc13d884b70f1ed8aad3615380 not found: ID does not exist"
Jul 02 13:54:36.104820 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.104760    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7c747fbc6-sxqt7"]
Jul 02 13:54:36.108269 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.108250    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-7c747fbc6-sxqt7"]
Jul 02 13:54:36.902855 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:36.902824    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc1826ae-bd90-47bb-aa97-62a8439972d7" path="/var/lib/kubelet/pods/bc1826ae-bd90-47bb-aa97-62a8439972d7/volumes"
Jul 02 13:54:37.094550 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:37.094518    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l" event={"ID":"69470978-4c8f-455c-9f12-154c6a6f64fd","Type":"ContainerStarted","Data":"fdb809baf138594d71a747229ba2497eb0eaaa1e3203505a8e8a1896766e5060"}
Jul 02 13:54:37.094764 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:37.094610    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l"
Jul 02 13:54:37.112427 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:54:37.112373    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l" podStartSLOduration=1.822834968 podStartE2EDuration="2.112356779s" totalImagesPullingTime="289.521811ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:54:37.110842642 +0000 UTC m=+562.615967139" watchObservedRunningTime="2026-07-02 13:54:37.112356779 +0000 UTC m=+562.617481282"
Jul 02 13:55:04.859380 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:04.859337    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:55:08.105881 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:08.105852    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7c747fbc6-hrc9l"
Jul 02 13:55:09.112019 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:09.111975    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:55:09.246083 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:09.246050    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:55:09.375700 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:09.375618    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:55:09.477665 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:09.477631    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:55:09.832375 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:09.832347    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:55:09.916530 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:09.916499    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:55:14.864526 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:14.864476    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 13:55:14.865266 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:14.865247    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 13:55:14.871555 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:14.871531    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 13:55:14.872265 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:55:14.872243    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 13:56:13.756884 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:56:13.756843    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:56:13.858132 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:56:13.858102    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:56:45.439851 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:56:45.439818    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:57:06.349309 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:06.349228    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:57:06.444152 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:06.444096    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:57:20.161174 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:20.161140    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:57:21.652611 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:21.652578    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:57:31.636191 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.636115    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:57:31.795511 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.795460    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"]
Jul 02 13:57:31.795801 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.795787    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc1826ae-bd90-47bb-aa97-62a8439972d7" containerName="manager"
Jul 02 13:57:31.795843 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.795803    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc1826ae-bd90-47bb-aa97-62a8439972d7" containerName="manager"
Jul 02 13:57:31.795877 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.795862    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc1826ae-bd90-47bb-aa97-62a8439972d7" containerName="manager"
Jul 02 13:57:31.799186 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.799165    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:31.803335 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.803315    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-predictor-serving-cert\""
Jul 02 13:57:31.803335 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.803330    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 13:57:31.803527 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.803493    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 02 13:57:31.803605 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.803556    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 13:57:31.803718 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.803693    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 02 13:57:31.803718 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.803696    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-dg2z6\""
Jul 02 13:57:31.803890 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.803833    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-kube-rbac-proxy-sar-config\""
Jul 02 13:57:31.813623 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.813595    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"]
Jul 02 13:57:31.924773 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.924687    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:57:31.932813 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.932783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c81fe33c-e17e-4bd5-8df9-d5169e139545-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:31.932947 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.932838    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c81fe33c-e17e-4bd5-8df9-d5169e139545-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:31.932947 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.932910    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:31.933020 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.932951    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:31.933020 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:31.932989    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59w4f\" (UniqueName: \"kubernetes.io/projected/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-api-access-59w4f\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.033880 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.033842    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c81fe33c-e17e-4bd5-8df9-d5169e139545-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.034055 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.034010    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c81fe33c-e17e-4bd5-8df9-d5169e139545-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.034113 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.034068    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.034174 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.034108    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.034174 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.034159    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59w4f\" (UniqueName: \"kubernetes.io/projected/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-api-access-59w4f\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.034522 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.034458    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c81fe33c-e17e-4bd5-8df9-d5169e139545-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.034774 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.034752    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.034774 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.034767    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.036324 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.036303    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c81fe33c-e17e-4bd5-8df9-d5169e139545-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.044744 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.044722    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59w4f\" (UniqueName: \"kubernetes.io/projected/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-api-access-59w4f\") pod \"isvc-sklearn-batcher-predictor-5b79f65797-fwhc5\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.110226 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.110188    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:32.252187 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.252154    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"]
Jul 02 13:57:32.255580 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:57:32.255550    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81fe33c_e17e_4bd5_8df9_d5169e139545.slice/crio-10cb3a248408ce84d16c1de9f3b31e0b9d3848e4bf56df9d00c5dc352657b573 WatchSource:0}: Error finding container 10cb3a248408ce84d16c1de9f3b31e0b9d3848e4bf56df9d00c5dc352657b573: Status 404 returned error can't find the container with id 10cb3a248408ce84d16c1de9f3b31e0b9d3848e4bf56df9d00c5dc352657b573
Jul 02 13:57:32.257954 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.257938    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 13:57:32.528494 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.528388    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:57:32.749621 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:32.749588    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" event={"ID":"c81fe33c-e17e-4bd5-8df9-d5169e139545","Type":"ContainerStarted","Data":"10cb3a248408ce84d16c1de9f3b31e0b9d3848e4bf56df9d00c5dc352657b573"}
Jul 02 13:57:35.768666 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:35.768631    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" event={"ID":"c81fe33c-e17e-4bd5-8df9-d5169e139545","Type":"ContainerStarted","Data":"cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65"}
Jul 02 13:57:39.785972 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:39.785891    2567 generic.go:356] "Generic (PLEG): container finished" podID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerID="cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65" exitCode=0
Jul 02 13:57:39.785972 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:39.785958    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" event={"ID":"c81fe33c-e17e-4bd5-8df9-d5169e139545","Type":"ContainerDied","Data":"cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65"}
Jul 02 13:57:53.851650 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:53.851607    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" event={"ID":"c81fe33c-e17e-4bd5-8df9-d5169e139545","Type":"ContainerStarted","Data":"96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097"}
Jul 02 13:57:54.137438 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:54.137349    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:57:55.863704 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:55.863664    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" event={"ID":"c81fe33c-e17e-4bd5-8df9-d5169e139545","Type":"ContainerStarted","Data":"958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea"}
Jul 02 13:57:58.879560 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:58.879525    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" event={"ID":"c81fe33c-e17e-4bd5-8df9-d5169e139545","Type":"ContainerStarted","Data":"40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456"}
Jul 02 13:57:58.879995 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:58.879770    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:58.879995 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:58.879838    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:58.879995 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:58.879876    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:57:58.881363 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:58.881338    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:57:58.882012 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:58.881991    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:57:58.907287 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:58.907238    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podStartSLOduration=2.361783107 podStartE2EDuration="27.907222534s" totalImagesPullingTime="21.794733979s" totalInitContainerRuntime="3.750705448s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:57:31 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:57:58.905627822 +0000 UTC m=+764.410752342" watchObservedRunningTime="2026-07-02 13:57:58.907222534 +0000 UTC m=+764.412347107"
Jul 02 13:57:59.883364 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:59.883316    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:57:59.883834 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:57:59.883810    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:58:04.887615 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:04.887582    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:58:04.888158 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:04.888117    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:58:04.888534 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:04.888505    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:58:14.888609 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:14.888570    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:58:14.889073 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:14.889048    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:58:24.888251 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:24.888211    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:58:24.888726 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:24.888662    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:58:34.888293 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:34.888247    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:58:34.888747 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:34.888723    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:58:44.889022 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:44.888926    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:58:44.889572 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:44.889375    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:58:54.888911 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:54.888869    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:58:54.889442 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:58:54.889419    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:59:04.888048 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:04.888000    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:59:04.888473 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:04.888362    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:59:14.889575 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:14.889544    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:59:14.890059 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:14.889687    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:59:18.579582 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:18.579544    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:18.679933 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:18.679898    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:18.919544 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:18.919454    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:19.514916 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:19.514890    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:19.602333 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:19.602306    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:26.834247 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.834185    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:26.846037 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.846000    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"]
Jul 02 13:59:26.846338 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.846307    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" containerID="cri-o://96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097" gracePeriod=30
Jul 02 13:59:26.846548 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.846345    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" containerID="cri-o://40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456" gracePeriod=30
Jul 02 13:59:26.846652 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.846624    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kube-rbac-proxy" containerID="cri-o://958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea" gracePeriod=30
Jul 02 13:59:26.870600 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:59:26.870524    2567 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81fe33c_e17e_4bd5_8df9_d5169e139545.slice/crio-958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea.scope\": RecentStats: unable to find data in memory cache]"
Jul 02 13:59:26.880739 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.880705    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"]
Jul 02 13:59:26.884338 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.884318    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:26.886707 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.886681    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-predictor-serving-cert\""
Jul 02 13:59:26.886831 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.886691    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-kube-rbac-proxy-sar-config\""
Jul 02 13:59:26.896668 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.896642    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"]
Jul 02 13:59:26.919396 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.919365    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:26.983082 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.983047    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8h4d\" (UniqueName: \"kubernetes.io/projected/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-api-access-j8h4d\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:26.983306 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.983091    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd2a0dd6-9a57-4964-960f-8c27112242f1-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:26.983306 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.983190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:26.983306 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.983236    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd2a0dd6-9a57-4964-960f-8c27112242f1-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:26.983306 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:26.983287    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.084401 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.084312    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd2a0dd6-9a57-4964-960f-8c27112242f1-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.084401 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.084356    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.084640 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.084402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8h4d\" (UniqueName: \"kubernetes.io/projected/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-api-access-j8h4d\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.084640 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.084429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd2a0dd6-9a57-4964-960f-8c27112242f1-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.084640 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:59:27.084574    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-serving-cert: secret "isvc-sklearn-batcher-custom-predictor-serving-cert" not found
Jul 02 13:59:27.084640 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.084616    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.084640 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:59:27.084637    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd2a0dd6-9a57-4964-960f-8c27112242f1-proxy-tls podName:dd2a0dd6-9a57-4964-960f-8c27112242f1 nodeName:}" failed. No retries permitted until 2026-07-02 13:59:27.584616572 +0000 UTC m=+853.089741053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/dd2a0dd6-9a57-4964-960f-8c27112242f1-proxy-tls") pod "isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" (UID: "dd2a0dd6-9a57-4964-960f-8c27112242f1") : secret "isvc-sklearn-batcher-custom-predictor-serving-cert" not found
Jul 02 13:59:27.084939 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.084807    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd2a0dd6-9a57-4964-960f-8c27112242f1-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.085107 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.085076    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.085252 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.085231    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.093301 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.093278    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8h4d\" (UniqueName: \"kubernetes.io/projected/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-api-access-j8h4d\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.166090 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.166047    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:27.225174 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.225141    2567 generic.go:356] "Generic (PLEG): container finished" podID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerID="958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea" exitCode=2
Jul 02 13:59:27.225338 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.225212    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" event={"ID":"c81fe33c-e17e-4bd5-8df9-d5169e139545","Type":"ContainerDied","Data":"958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea"}
Jul 02 13:59:27.588493 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.588451    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd2a0dd6-9a57-4964-960f-8c27112242f1-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.590989 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.590960    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd2a0dd6-9a57-4964-960f-8c27112242f1-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.794734 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.794687    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:27.928796 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:27.928749    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"]
Jul 02 13:59:27.931352 ip-10-0-137-226 kubenswrapper[2567]: W0702 13:59:27.931327    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd2a0dd6_9a57_4964_960f_8c27112242f1.slice/crio-5779075bab1b58abf238ee6fa8f8fae87c48e0a26c7adaaf1c7702ccf79fde0d WatchSource:0}: Error finding container 5779075bab1b58abf238ee6fa8f8fae87c48e0a26c7adaaf1c7702ccf79fde0d: Status 404 returned error can't find the container with id 5779075bab1b58abf238ee6fa8f8fae87c48e0a26c7adaaf1c7702ccf79fde0d
Jul 02 13:59:28.231072 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:28.230993    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" event={"ID":"dd2a0dd6-9a57-4964-960f-8c27112242f1","Type":"ContainerStarted","Data":"52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78"}
Jul 02 13:59:28.231072 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:28.231058    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" event={"ID":"dd2a0dd6-9a57-4964-960f-8c27112242f1","Type":"ContainerStarted","Data":"5779075bab1b58abf238ee6fa8f8fae87c48e0a26c7adaaf1c7702ccf79fde0d"}
Jul 02 13:59:29.883693 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:29.883640    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused"
Jul 02 13:59:29.986450 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:29.986392    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:30.045921 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:30.045884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:32.247597 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:32.247566    2567 generic.go:356] "Generic (PLEG): container finished" podID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerID="52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78" exitCode=0
Jul 02 13:59:32.248029 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:32.247649    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" event={"ID":"dd2a0dd6-9a57-4964-960f-8c27112242f1","Type":"ContainerDied","Data":"52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78"}
Jul 02 13:59:32.253542 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:32.253518    2567 generic.go:356] "Generic (PLEG): container finished" podID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerID="96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097" exitCode=0
Jul 02 13:59:32.253636 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:32.253554    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" event={"ID":"c81fe33c-e17e-4bd5-8df9-d5169e139545","Type":"ContainerDied","Data":"96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097"}
Jul 02 13:59:33.260575 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:33.260540    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" event={"ID":"dd2a0dd6-9a57-4964-960f-8c27112242f1","Type":"ContainerStarted","Data":"60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f"}
Jul 02 13:59:33.261100 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:33.260610    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" event={"ID":"dd2a0dd6-9a57-4964-960f-8c27112242f1","Type":"ContainerStarted","Data":"2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861"}
Jul 02 13:59:33.261100 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:33.260636    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" event={"ID":"dd2a0dd6-9a57-4964-960f-8c27112242f1","Type":"ContainerStarted","Data":"60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709"}
Jul 02 13:59:33.261100 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:33.260913    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:33.261100 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:33.260972    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:33.262442 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:33.262414    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 13:59:33.287160 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:33.287114    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podStartSLOduration=3.836993358 podStartE2EDuration="7.287098101s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.450104743s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:59:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:59:33.284771705 +0000 UTC m=+858.789896208" watchObservedRunningTime="2026-07-02 13:59:33.287098101 +0000 UTC m=+858.792222671"
Jul 02 13:59:34.265007 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:34.264976    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:34.265007 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:34.265002    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 13:59:34.266305 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:34.266278    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:59:34.884551 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:34.884510    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused"
Jul 02 13:59:34.888872 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:34.888841    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:59:34.889266 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:34.889244    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:59:35.269199 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:35.269107    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 13:59:35.269631 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:35.269477    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:59:39.884638 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:39.884594    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused"
Jul 02 13:59:39.885046 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:39.884737    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:59:40.273545 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:40.273462    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 13:59:40.274028 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:40.273994    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 13:59:40.274397 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:40.274378    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:59:44.883962 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:44.883911    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused"
Jul 02 13:59:44.888288 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:44.888253    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:59:44.888643 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:44.888617    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:59:45.854854 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:45.854819    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:45.911073 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:45.911041    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:49.883986 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:49.883942    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused"
Jul 02 13:59:50.274823 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:50.274723    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 13:59:50.275212 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:50.275187    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:59:54.884294 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:54.884247    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused"
Jul 02 13:59:54.888721 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:54.888676    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Jul 02 13:59:54.888872 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:54.888831    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:59:54.889028 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:54.889007    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:59:54.889123 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:54.889109    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:59:57.002571 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.002535    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:59:57.143657 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.143573    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59w4f\" (UniqueName: \"kubernetes.io/projected/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-api-access-59w4f\") pod \"c81fe33c-e17e-4bd5-8df9-d5169e139545\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") "
Jul 02 13:59:57.143657 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.143611    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c81fe33c-e17e-4bd5-8df9-d5169e139545-proxy-tls\") pod \"c81fe33c-e17e-4bd5-8df9-d5169e139545\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") "
Jul 02 13:59:57.143657 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.143648    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c81fe33c-e17e-4bd5-8df9-d5169e139545-kserve-provision-location\") pod \"c81fe33c-e17e-4bd5-8df9-d5169e139545\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") "
Jul 02 13:59:57.143891 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.143696    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-rbac-proxy-sar-config\") pod \"c81fe33c-e17e-4bd5-8df9-d5169e139545\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") "
Jul 02 13:59:57.143891 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.143794    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-cabundle-cert\") pod \"c81fe33c-e17e-4bd5-8df9-d5169e139545\" (UID: \"c81fe33c-e17e-4bd5-8df9-d5169e139545\") "
Jul 02 13:59:57.144021 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.143994    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c81fe33c-e17e-4bd5-8df9-d5169e139545-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c81fe33c-e17e-4bd5-8df9-d5169e139545" (UID: "c81fe33c-e17e-4bd5-8df9-d5169e139545"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 13:59:57.144151 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.144124    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c81fe33c-e17e-4bd5-8df9-d5169e139545" (UID: "c81fe33c-e17e-4bd5-8df9-d5169e139545"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:59:57.144223 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.144137    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "c81fe33c-e17e-4bd5-8df9-d5169e139545" (UID: "c81fe33c-e17e-4bd5-8df9-d5169e139545"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:59:57.145911 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.145883    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-api-access-59w4f" (OuterVolumeSpecName: "kube-api-access-59w4f") pod "c81fe33c-e17e-4bd5-8df9-d5169e139545" (UID: "c81fe33c-e17e-4bd5-8df9-d5169e139545"). InnerVolumeSpecName "kube-api-access-59w4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:59:57.146017 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.145886    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c81fe33c-e17e-4bd5-8df9-d5169e139545-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c81fe33c-e17e-4bd5-8df9-d5169e139545" (UID: "c81fe33c-e17e-4bd5-8df9-d5169e139545"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:59:57.244965 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.244928    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-cabundle-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:59:57.244965 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.244960    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59w4f\" (UniqueName: \"kubernetes.io/projected/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-api-access-59w4f\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:59:57.244965 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.244970    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c81fe33c-e17e-4bd5-8df9-d5169e139545-proxy-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:59:57.245202 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.244979    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c81fe33c-e17e-4bd5-8df9-d5169e139545-kserve-provision-location\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:59:57.245202 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.244988    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c81fe33c-e17e-4bd5-8df9-d5169e139545-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 13:59:57.360855 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.360814    2567 generic.go:356] "Generic (PLEG): container finished" podID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerID="40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456" exitCode=0
Jul 02 13:59:57.361039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.360876    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" event={"ID":"c81fe33c-e17e-4bd5-8df9-d5169e139545","Type":"ContainerDied","Data":"40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456"}
Jul 02 13:59:57.361039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.360890    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"
Jul 02 13:59:57.361039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.360948    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5" event={"ID":"c81fe33c-e17e-4bd5-8df9-d5169e139545","Type":"ContainerDied","Data":"10cb3a248408ce84d16c1de9f3b31e0b9d3848e4bf56df9d00c5dc352657b573"}
Jul 02 13:59:57.361039 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.360981    2567 scope.go:117] "RemoveContainer" containerID="40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456"
Jul 02 13:59:57.370635 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.370616    2567 scope.go:117] "RemoveContainer" containerID="958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea"
Jul 02 13:59:57.379146 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.379128    2567 scope.go:117] "RemoveContainer" containerID="96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097"
Jul 02 13:59:57.383966 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.383943    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"]
Jul 02 13:59:57.387786 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.387754    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-5b79f65797-fwhc5"]
Jul 02 13:59:57.388797 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.388779    2567 scope.go:117] "RemoveContainer" containerID="cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65"
Jul 02 13:59:57.396750 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.396732    2567 scope.go:117] "RemoveContainer" containerID="40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456"
Jul 02 13:59:57.397011 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:59:57.396989    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456\": container with ID starting with 40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456 not found: ID does not exist" containerID="40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456"
Jul 02 13:59:57.397087 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.397019    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456"} err="failed to get container status \"40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456\": rpc error: code = NotFound desc = could not find container \"40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456\": container with ID starting with 40f712b50da213eb25f5499cb381fe063d9100b7ba5c96d2e44637458cb4f456 not found: ID does not exist"
Jul 02 13:59:57.397087 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.397039    2567 scope.go:117] "RemoveContainer" containerID="958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea"
Jul 02 13:59:57.397282 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:59:57.397266    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea\": container with ID starting with 958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea not found: ID does not exist" containerID="958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea"
Jul 02 13:59:57.397328 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.397286    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea"} err="failed to get container status \"958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea\": rpc error: code = NotFound desc = could not find container \"958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea\": container with ID starting with 958311627d736f5ae68eea764449a4067ebe6e246dbe3285fdcea3653e99e1ea not found: ID does not exist"
Jul 02 13:59:57.397328 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.397299    2567 scope.go:117] "RemoveContainer" containerID="96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097"
Jul 02 13:59:57.397551 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:59:57.397533    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097\": container with ID starting with 96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097 not found: ID does not exist" containerID="96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097"
Jul 02 13:59:57.397617 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.397558    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097"} err="failed to get container status \"96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097\": rpc error: code = NotFound desc = could not find container \"96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097\": container with ID starting with 96e4cb81e6b99f513a995940784886002e2f85e106a54f4f03d0e3e3b7ae4097 not found: ID does not exist"
Jul 02 13:59:57.397617 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.397581    2567 scope.go:117] "RemoveContainer" containerID="cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65"
Jul 02 13:59:57.397806 ip-10-0-137-226 kubenswrapper[2567]: E0702 13:59:57.397791    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65\": container with ID starting with cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65 not found: ID does not exist" containerID="cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65"
Jul 02 13:59:57.397889 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:57.397808    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65"} err="failed to get container status \"cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65\": rpc error: code = NotFound desc = could not find container \"cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65\": container with ID starting with cea1a825458611cda1cc78509e052b1ad4e8bb4e999f46c7e855b1454c2c1a65 not found: ID does not exist"
Jul 02 13:59:58.903473 ip-10-0-137-226 kubenswrapper[2567]: I0702 13:59:58.903442    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" path="/var/lib/kubelet/pods/c81fe33c-e17e-4bd5-8df9-d5169e139545/volumes"
Jul 02 14:00:00.273965 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:00.273926    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 14:00:00.274445 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:00.274314    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:00:10.274327 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:10.274242    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 14:00:10.274738 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:10.274630    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:00:14.889788 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:14.889763    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 14:00:14.891498 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:14.891460    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 14:00:14.902096 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:14.902071    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 14:00:14.903322 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:14.903293    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 14:00:20.274110 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:20.274067    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 14:00:20.274551 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:20.274435    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:00:30.274874 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:30.274834    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 14:00:30.275377 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:30.275182    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:00:40.273965 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:40.273920    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 14:00:40.274419 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:40.274396    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:00:45.898826 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:45.898775    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 14:00:45.899281 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:45.899166    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:00:55.900148 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:55.900119    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 14:00:55.900633 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:00:55.900528    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 14:01:02.058587 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.058553    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:01:02.104988 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.104959    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"]
Jul 02 14:01:02.105314 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.105301    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent"
Jul 02 14:01:02.105360 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.105316    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent"
Jul 02 14:01:02.105360 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.105329    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container"
Jul 02 14:01:02.105360 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.105335    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container"
Jul 02 14:01:02.105360 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.105349    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kube-rbac-proxy"
Jul 02 14:01:02.105360 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.105356    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kube-rbac-proxy"
Jul 02 14:01:02.105545 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.105376    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="storage-initializer"
Jul 02 14:01:02.105545 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.105381    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="storage-initializer"
Jul 02 14:01:02.105545 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.105429    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kube-rbac-proxy"
Jul 02 14:01:02.105545 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.105442    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="kserve-container"
Jul 02 14:01:02.105545 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.105452    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c81fe33c-e17e-4bd5-8df9-d5169e139545" containerName="agent"
Jul 02 14:01:02.108512 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.108461    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:02.110922 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.110903    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-kube-rbac-proxy-sar-config\""
Jul 02 14:01:02.111340 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.111323    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-predictor-serving-cert\""
Jul 02 14:01:02.116850 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.116812    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"]
Jul 02 14:01:02.117110 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.117078    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" containerID="cri-o://60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709" gracePeriod=30
Jul 02 14:01:02.117180 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.117128    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" containerID="cri-o://60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f" gracePeriod=30
Jul 02 14:01:02.117239 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.117173    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kube-rbac-proxy" containerID="cri-o://2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861" gracePeriod=30
Jul 02 14:01:02.121378 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.121350    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"]
Jul 02 14:01:02.156448 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.156423    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:01:02.193649 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.193617    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz6q2\" (UniqueName: \"kubernetes.io/projected/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-api-access-hz6q2\") pod \"message-dumper-predictor-6575bcfdfd-jsfdr\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:02.193817 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.193686    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-jsfdr\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:02.193817 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.193711    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-jsfdr\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:02.256966 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.256938    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:01:02.294251 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.294217    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz6q2\" (UniqueName: \"kubernetes.io/projected/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-api-access-hz6q2\") pod \"message-dumper-predictor-6575bcfdfd-jsfdr\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:02.294417 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.294280    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-jsfdr\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:02.294417 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.294310    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-jsfdr\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:02.295189 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.295158    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-jsfdr\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:02.296772 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.296742    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-jsfdr\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:02.303221 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.303175    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz6q2\" (UniqueName: \"kubernetes.io/projected/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-api-access-hz6q2\") pod \"message-dumper-predictor-6575bcfdfd-jsfdr\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:02.422551 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.422452    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:02.559227 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.559203    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"]
Jul 02 14:01:02.561654 ip-10-0-137-226 kubenswrapper[2567]: W0702 14:01:02.561628    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92cfa623_e6fb_4676_a9d9_1ea1d08e0778.slice/crio-3b4aa3e19f205ace01540b5445ddb0fdeb8477879311f3f2ed4121aaf71638f6 WatchSource:0}: Error finding container 3b4aa3e19f205ace01540b5445ddb0fdeb8477879311f3f2ed4121aaf71638f6: Status 404 returned error can't find the container with id 3b4aa3e19f205ace01540b5445ddb0fdeb8477879311f3f2ed4121aaf71638f6
Jul 02 14:01:02.614669 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.614639    2567 generic.go:356] "Generic (PLEG): container finished" podID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerID="2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861" exitCode=2
Jul 02 14:01:02.614848 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.614715    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" event={"ID":"dd2a0dd6-9a57-4964-960f-8c27112242f1","Type":"ContainerDied","Data":"2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861"}
Jul 02 14:01:02.615820 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:02.615797    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr" event={"ID":"92cfa623-e6fb-4676-a9d9-1ea1d08e0778","Type":"ContainerStarted","Data":"3b4aa3e19f205ace01540b5445ddb0fdeb8477879311f3f2ed4121aaf71638f6"}
Jul 02 14:01:04.626658 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:04.626625    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr" event={"ID":"92cfa623-e6fb-4676-a9d9-1ea1d08e0778","Type":"ContainerStarted","Data":"230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5"}
Jul 02 14:01:04.627055 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:04.626690    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr" event={"ID":"92cfa623-e6fb-4676-a9d9-1ea1d08e0778","Type":"ContainerStarted","Data":"a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016"}
Jul 02 14:01:04.627055 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:04.626742    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:04.646078 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:04.646030    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr" podStartSLOduration=1.479370949 podStartE2EDuration="2.64601566s" totalImagesPullingTime="1.166644711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:01:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:01:04.644266507 +0000 UTC m=+950.149391044" watchObservedRunningTime="2026-07-02 14:01:04.64601566 +0000 UTC m=+950.151140164"
Jul 02 14:01:05.269194 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:05.269153    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused"
Jul 02 14:01:05.630682 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:05.630650    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:05.632419 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:05.632399    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:05.900123 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:05.900027    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:01:05.901315 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:05.901292    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 14:01:07.640286 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:07.640254    2567 generic.go:356] "Generic (PLEG): container finished" podID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerID="60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709" exitCode=0
Jul 02 14:01:07.640671 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:07.640324    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" event={"ID":"dd2a0dd6-9a57-4964-960f-8c27112242f1","Type":"ContainerDied","Data":"60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709"}
Jul 02 14:01:10.269818 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:10.269773    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused"
Jul 02 14:01:12.644773 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:12.644745    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:01:15.269332 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:15.269283    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused"
Jul 02 14:01:15.269737 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:15.269438    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 14:01:15.899161 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:15.899115    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:01:15.901449 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:15.901414    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 14:01:20.269694 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:20.269653    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused"
Jul 02 14:01:22.087797 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.087761    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:01:22.131728 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.131697    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"]
Jul 02 14:01:22.135251 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.135229    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.137664 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.137635    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-kube-rbac-proxy-sar-config\""
Jul 02 14:01:22.137786 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.137641    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-predictor-serving-cert\""
Jul 02 14:01:22.146322 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.146297    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"]
Jul 02 14:01:22.156686 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.156653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cxqz\" (UniqueName: \"kubernetes.io/projected/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-api-access-2cxqz\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.156843 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.156713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-proxy-tls\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.156843 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.156742    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-cabundle-cert\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.156843 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.156827    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.156986 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.156862    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kserve-provision-location\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.257994 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.257952    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-proxy-tls\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.257994 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.257992    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-cabundle-cert\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.258250 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.258030    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.258250 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.258051    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kserve-provision-location\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.258250 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.258081    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cxqz\" (UniqueName: \"kubernetes.io/projected/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-api-access-2cxqz\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.258250 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:01:22.258087    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-logger-predictor-serving-cert: secret "isvc-logger-predictor-serving-cert" not found
Jul 02 14:01:22.258250 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:01:22.258174    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-proxy-tls podName:de2c995d-e3fb-4639-bf9e-eae8fceb15e0 nodeName:}" failed. No retries permitted until 2026-07-02 14:01:22.758152192 +0000 UTC m=+968.263276678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-proxy-tls") pod "isvc-logger-predictor-547b85fd58-gxh5m" (UID: "de2c995d-e3fb-4639-bf9e-eae8fceb15e0") : secret "isvc-logger-predictor-serving-cert" not found
Jul 02 14:01:22.258575 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.258547    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kserve-provision-location\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.258775 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.258758    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.258813 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.258762    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-cabundle-cert\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.267261 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.267238    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cxqz\" (UniqueName: \"kubernetes.io/projected/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-api-access-2cxqz\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.350380 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.350281    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:01:22.762553 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.762450    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-proxy-tls\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:22.764811 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:22.764781    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-proxy-tls\") pod \"isvc-logger-predictor-547b85fd58-gxh5m\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:23.047383 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:23.047353    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:23.183021 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:23.182999    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"]
Jul 02 14:01:23.185199 ip-10-0-137-226 kubenswrapper[2567]: W0702 14:01:23.185171    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2c995d_e3fb_4639_bf9e_eae8fceb15e0.slice/crio-e8b2e92845731de92d754bd3e9f5d7b361b8019eb834408b2c374b8c6972c552 WatchSource:0}: Error finding container e8b2e92845731de92d754bd3e9f5d7b361b8019eb834408b2c374b8c6972c552: Status 404 returned error can't find the container with id e8b2e92845731de92d754bd3e9f5d7b361b8019eb834408b2c374b8c6972c552
Jul 02 14:01:23.705219 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:23.705182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" event={"ID":"de2c995d-e3fb-4639-bf9e-eae8fceb15e0","Type":"ContainerStarted","Data":"7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5"}
Jul 02 14:01:23.705419 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:23.705244    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" event={"ID":"de2c995d-e3fb-4639-bf9e-eae8fceb15e0","Type":"ContainerStarted","Data":"e8b2e92845731de92d754bd3e9f5d7b361b8019eb834408b2c374b8c6972c552"}
Jul 02 14:01:25.269937 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:25.269888    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused"
Jul 02 14:01:25.900099 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:25.900047    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:01:25.900273 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:25.900181    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 14:01:25.900604 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:25.900583    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:5000: connect: connection refused"
Jul 02 14:01:25.900681 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:25.900668    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 14:01:27.446774 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:27.446736    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:01:27.722496 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:27.722404    2567 generic.go:356] "Generic (PLEG): container finished" podID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerID="7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5" exitCode=0
Jul 02 14:01:27.722496 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:27.722444    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" event={"ID":"de2c995d-e3fb-4639-bf9e-eae8fceb15e0","Type":"ContainerDied","Data":"7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5"}
Jul 02 14:01:28.729897 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:28.729867    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" event={"ID":"de2c995d-e3fb-4639-bf9e-eae8fceb15e0","Type":"ContainerStarted","Data":"10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34"}
Jul 02 14:01:28.730266 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:28.729929    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" event={"ID":"de2c995d-e3fb-4639-bf9e-eae8fceb15e0","Type":"ContainerStarted","Data":"95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62"}
Jul 02 14:01:28.730266 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:28.729955    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" event={"ID":"de2c995d-e3fb-4639-bf9e-eae8fceb15e0","Type":"ContainerStarted","Data":"cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921"}
Jul 02 14:01:28.730266 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:28.730232    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:28.730382 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:28.730365    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:28.731620 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:28.731592    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:01:28.751966 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:28.751908    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podStartSLOduration=3.189042848 podStartE2EDuration="6.751887151s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.562844303s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:01:28.751602363 +0000 UTC m=+974.256726895" watchObservedRunningTime="2026-07-02 14:01:28.751887151 +0000 UTC m=+974.257011660"
Jul 02 14:01:29.734344 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:29.734304    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:29.734756 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:29.734500    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:01:29.735553 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:29.735525    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:01:30.269283 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:30.269244    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused"
Jul 02 14:01:30.738150 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:30.738110    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:01:30.738621 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:30.738534    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:01:32.268157 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.268128    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 14:01:32.342275 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.342236    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd2a0dd6-9a57-4964-960f-8c27112242f1-proxy-tls\") pod \"dd2a0dd6-9a57-4964-960f-8c27112242f1\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") "
Jul 02 14:01:32.342456 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.342308    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-rbac-proxy-sar-config\") pod \"dd2a0dd6-9a57-4964-960f-8c27112242f1\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") "
Jul 02 14:01:32.342456 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.342331    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd2a0dd6-9a57-4964-960f-8c27112242f1-kserve-provision-location\") pod \"dd2a0dd6-9a57-4964-960f-8c27112242f1\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") "
Jul 02 14:01:32.342456 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.342403    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-cabundle-cert\") pod \"dd2a0dd6-9a57-4964-960f-8c27112242f1\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") "
Jul 02 14:01:32.342456 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.342434    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8h4d\" (UniqueName: \"kubernetes.io/projected/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-api-access-j8h4d\") pod \"dd2a0dd6-9a57-4964-960f-8c27112242f1\" (UID: \"dd2a0dd6-9a57-4964-960f-8c27112242f1\") "
Jul 02 14:01:32.342716 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.342692    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd2a0dd6-9a57-4964-960f-8c27112242f1-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "dd2a0dd6-9a57-4964-960f-8c27112242f1" (UID: "dd2a0dd6-9a57-4964-960f-8c27112242f1"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:01:32.342773 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.342689    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "dd2a0dd6-9a57-4964-960f-8c27112242f1" (UID: "dd2a0dd6-9a57-4964-960f-8c27112242f1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:01:32.342832 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.342791    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "dd2a0dd6-9a57-4964-960f-8c27112242f1" (UID: "dd2a0dd6-9a57-4964-960f-8c27112242f1"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:01:32.344587 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.344565    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-api-access-j8h4d" (OuterVolumeSpecName: "kube-api-access-j8h4d") pod "dd2a0dd6-9a57-4964-960f-8c27112242f1" (UID: "dd2a0dd6-9a57-4964-960f-8c27112242f1"). InnerVolumeSpecName "kube-api-access-j8h4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:01:32.344688 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.344664    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd2a0dd6-9a57-4964-960f-8c27112242f1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "dd2a0dd6-9a57-4964-960f-8c27112242f1" (UID: "dd2a0dd6-9a57-4964-960f-8c27112242f1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:01:32.443641 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.443548    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-cabundle-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:01:32.443641 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.443578    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8h4d\" (UniqueName: \"kubernetes.io/projected/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-api-access-j8h4d\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:01:32.443641 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.443590    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd2a0dd6-9a57-4964-960f-8c27112242f1-proxy-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:01:32.443641 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.443599    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd2a0dd6-9a57-4964-960f-8c27112242f1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:01:32.443641 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.443608    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd2a0dd6-9a57-4964-960f-8c27112242f1-kserve-provision-location\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:01:32.749174 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.749089    2567 generic.go:356] "Generic (PLEG): container finished" podID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerID="60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f" exitCode=0
Jul 02 14:01:32.749174 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.749165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" event={"ID":"dd2a0dd6-9a57-4964-960f-8c27112242f1","Type":"ContainerDied","Data":"60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f"}
Jul 02 14:01:32.749334 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.749204    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"
Jul 02 14:01:32.749334 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.749223    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r" event={"ID":"dd2a0dd6-9a57-4964-960f-8c27112242f1","Type":"ContainerDied","Data":"5779075bab1b58abf238ee6fa8f8fae87c48e0a26c7adaaf1c7702ccf79fde0d"}
Jul 02 14:01:32.749334 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.749254    2567 scope.go:117] "RemoveContainer" containerID="60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f"
Jul 02 14:01:32.760328 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.760150    2567 scope.go:117] "RemoveContainer" containerID="2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861"
Jul 02 14:01:32.768260 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.768226    2567 scope.go:117] "RemoveContainer" containerID="60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709"
Jul 02 14:01:32.775917 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.775896    2567 scope.go:117] "RemoveContainer" containerID="52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78"
Jul 02 14:01:32.777448 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.777427    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"]
Jul 02 14:01:32.783264 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.783243    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-6b699544d4-5x59r"]
Jul 02 14:01:32.784016 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.783993    2567 scope.go:117] "RemoveContainer" containerID="60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f"
Jul 02 14:01:32.784259 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:01:32.784243    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f\": container with ID starting with 60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f not found: ID does not exist" containerID="60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f"
Jul 02 14:01:32.784320 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.784270    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f"} err="failed to get container status \"60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f\": rpc error: code = NotFound desc = could not find container \"60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f\": container with ID starting with 60057594868b4b7947f7572ed48fa0cff78a455dfe7b05e0e5558bb1ea38be5f not found: ID does not exist"
Jul 02 14:01:32.784320 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.784288    2567 scope.go:117] "RemoveContainer" containerID="2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861"
Jul 02 14:01:32.784552 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:01:32.784534    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861\": container with ID starting with 2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861 not found: ID does not exist" containerID="2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861"
Jul 02 14:01:32.784633 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.784561    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861"} err="failed to get container status \"2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861\": rpc error: code = NotFound desc = could not find container \"2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861\": container with ID starting with 2137ccc2b07011b26d8c64a2f294d4d323071494400d7d8df98d66593d5a3861 not found: ID does not exist"
Jul 02 14:01:32.784633 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.784585    2567 scope.go:117] "RemoveContainer" containerID="60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709"
Jul 02 14:01:32.784833 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:01:32.784816    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709\": container with ID starting with 60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709 not found: ID does not exist" containerID="60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709"
Jul 02 14:01:32.784872 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.784838    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709"} err="failed to get container status \"60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709\": rpc error: code = NotFound desc = could not find container \"60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709\": container with ID starting with 60fe776d8c9500a4f3a072c3cde6e4daa0fde178d5852b39291f288cf0bd2709 not found: ID does not exist"
Jul 02 14:01:32.784872 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.784852    2567 scope.go:117] "RemoveContainer" containerID="52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78"
Jul 02 14:01:32.785080 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:01:32.785057    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78\": container with ID starting with 52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78 not found: ID does not exist" containerID="52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78"
Jul 02 14:01:32.785136 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.785087    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78"} err="failed to get container status \"52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78\": rpc error: code = NotFound desc = could not find container \"52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78\": container with ID starting with 52640d8f2f06a698e18130a38ef510b6c103e37ec16c754adf148c3414c31c78 not found: ID does not exist"
Jul 02 14:01:32.904180 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:32.904147    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" path="/var/lib/kubelet/pods/dd2a0dd6-9a57-4964-960f-8c27112242f1/volumes"
Jul 02 14:01:35.742293 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:35.742263    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:01:35.743030 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:35.742992    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:01:35.743252 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:35.743230    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:01:45.742899 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:45.742804    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:01:45.743335 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:45.743310    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:01:55.743512 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:55.743381    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:01:55.743910 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:01:55.743886    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:02:05.743262 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:05.743222    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:02:05.743801 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:05.743682    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:02:15.743757 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:15.743712    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:02:15.744165 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:15.744102    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:02:25.743343 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:25.743297    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:02:25.743809 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:25.743783    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:02:35.742963 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:35.742918    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:02:35.743363 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:35.743306    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:02:45.744139 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:45.744104    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:02:45.744677 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:45.744543    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:02:57.098769 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.098736    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-predictor-6575bcfdfd-jsfdr_92cfa623-e6fb-4676-a9d9-1ea1d08e0778/kserve-container/0.log"
Jul 02 14:02:57.377051 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.376958    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:02:57.429716 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.427432    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"]
Jul 02 14:02:57.429716 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.428896    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" containerID="cri-o://cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921" gracePeriod=30
Jul 02 14:02:57.429716 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.429076    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" containerID="cri-o://10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34" gracePeriod=30
Jul 02 14:02:57.429716 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.429170    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kube-rbac-proxy" containerID="cri-o://95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62" gracePeriod=30
Jul 02 14:02:57.447858 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.447831    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"]
Jul 02 14:02:57.448426 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.448402    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kube-rbac-proxy"
Jul 02 14:02:57.448611 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.448584    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kube-rbac-proxy"
Jul 02 14:02:57.448725 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.448619    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container"
Jul 02 14:02:57.448725 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.448631    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container"
Jul 02 14:02:57.448725 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.448659    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="storage-initializer"
Jul 02 14:02:57.448725 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.448667    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="storage-initializer"
Jul 02 14:02:57.448725 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.448685    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent"
Jul 02 14:02:57.448725 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.448692    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent"
Jul 02 14:02:57.449121 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.448778    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kube-rbac-proxy"
Jul 02 14:02:57.449121 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.448792    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="agent"
Jul 02 14:02:57.449121 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.448804    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd2a0dd6-9a57-4964-960f-8c27112242f1" containerName="kserve-container"
Jul 02 14:02:57.452452 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.452430    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.455020 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.454991    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-predictor-serving-cert\""
Jul 02 14:02:57.455313 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.455294    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-kube-rbac-proxy-sar-config\""
Jul 02 14:02:57.465729 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.465393    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"]
Jul 02 14:02:57.475063 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.475041    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"]
Jul 02 14:02:57.475506 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.475470    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr" podUID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" containerName="kube-rbac-proxy" containerID="cri-o://230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5" gracePeriod=30
Jul 02 14:02:57.475751 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.475428    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr" podUID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" containerName="kserve-container" containerID="cri-o://a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016" gracePeriod=30
Jul 02 14:02:57.529516 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.529466    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.529711 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.529539    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.529711 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.529611    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.529711 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.529663    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pl9q\" (UniqueName: \"kubernetes.io/projected/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-api-access-2pl9q\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.529711 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.529688    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b805e4b6-7c72-4b54-8f2e-5164ac246aec-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.630379 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.630293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b805e4b6-7c72-4b54-8f2e-5164ac246aec-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.630379 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.630360    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.630641 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.630411    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.630641 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.630457    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.630641 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:02:57.630512    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v1-predictor-serving-cert: secret "isvc-autogluon-v1-predictor-serving-cert" not found
Jul 02 14:02:57.630641 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:02:57.630590    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b805e4b6-7c72-4b54-8f2e-5164ac246aec-proxy-tls podName:b805e4b6-7c72-4b54-8f2e-5164ac246aec nodeName:}" failed. No retries permitted until 2026-07-02 14:02:58.130568668 +0000 UTC m=+1063.635693151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b805e4b6-7c72-4b54-8f2e-5164ac246aec-proxy-tls") pod "isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" (UID: "b805e4b6-7c72-4b54-8f2e-5164ac246aec") : secret "isvc-autogluon-v1-predictor-serving-cert" not found
Jul 02 14:02:57.630862 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.630515    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pl9q\" (UniqueName: \"kubernetes.io/projected/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-api-access-2pl9q\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.630862 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.630846    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.631117 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.631093    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.631195 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.631128    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.639183 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.639149    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pl9q\" (UniqueName: \"kubernetes.io/projected/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-api-access-2pl9q\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:57.641222 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.641194    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr" podUID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.26:8643/healthz\": dial tcp 10.134.0.26:8643: connect: connection refused"
Jul 02 14:02:57.733939 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.733906    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:02:57.833248 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.833209    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-rbac-proxy-sar-config\") pod \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") "
Jul 02 14:02:57.833248 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.833257    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-proxy-tls\") pod \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") "
Jul 02 14:02:57.833536 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.833277    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz6q2\" (UniqueName: \"kubernetes.io/projected/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-api-access-hz6q2\") pod \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\" (UID: \"92cfa623-e6fb-4676-a9d9-1ea1d08e0778\") "
Jul 02 14:02:57.833748 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.833713    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "92cfa623-e6fb-4676-a9d9-1ea1d08e0778" (UID: "92cfa623-e6fb-4676-a9d9-1ea1d08e0778"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:02:57.835514 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.835469    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "92cfa623-e6fb-4676-a9d9-1ea1d08e0778" (UID: "92cfa623-e6fb-4676-a9d9-1ea1d08e0778"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:02:57.835514 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.835504    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-api-access-hz6q2" (OuterVolumeSpecName: "kube-api-access-hz6q2") pod "92cfa623-e6fb-4676-a9d9-1ea1d08e0778" (UID: "92cfa623-e6fb-4676-a9d9-1ea1d08e0778"). InnerVolumeSpecName "kube-api-access-hz6q2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:02:57.934934 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.934843    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:02:57.934934 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.934877    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-proxy-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:02:57.934934 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:57.934886    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz6q2\" (UniqueName: \"kubernetes.io/projected/92cfa623-e6fb-4676-a9d9-1ea1d08e0778-kube-api-access-hz6q2\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:02:58.106716 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.106685    2567 generic.go:356] "Generic (PLEG): container finished" podID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" containerID="230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5" exitCode=2
Jul 02 14:02:58.106716 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.106711    2567 generic.go:356] "Generic (PLEG): container finished" podID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" containerID="a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016" exitCode=2
Jul 02 14:02:58.107261 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.106749    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"
Jul 02 14:02:58.107261 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.106768    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr" event={"ID":"92cfa623-e6fb-4676-a9d9-1ea1d08e0778","Type":"ContainerDied","Data":"230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5"}
Jul 02 14:02:58.107261 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.106833    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr" event={"ID":"92cfa623-e6fb-4676-a9d9-1ea1d08e0778","Type":"ContainerDied","Data":"a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016"}
Jul 02 14:02:58.107261 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.106866    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr" event={"ID":"92cfa623-e6fb-4676-a9d9-1ea1d08e0778","Type":"ContainerDied","Data":"3b4aa3e19f205ace01540b5445ddb0fdeb8477879311f3f2ed4121aaf71638f6"}
Jul 02 14:02:58.107261 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.106879    2567 scope.go:117] "RemoveContainer" containerID="230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5"
Jul 02 14:02:58.109468 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.109443    2567 generic.go:356] "Generic (PLEG): container finished" podID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerID="95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62" exitCode=2
Jul 02 14:02:58.109619 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.109507    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" event={"ID":"de2c995d-e3fb-4639-bf9e-eae8fceb15e0","Type":"ContainerDied","Data":"95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62"}
Jul 02 14:02:58.115811 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.115795    2567 scope.go:117] "RemoveContainer" containerID="a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016"
Jul 02 14:02:58.123252 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.123237    2567 scope.go:117] "RemoveContainer" containerID="230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5"
Jul 02 14:02:58.123519 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:02:58.123498    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5\": container with ID starting with 230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5 not found: ID does not exist" containerID="230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5"
Jul 02 14:02:58.123610 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.123525    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5"} err="failed to get container status \"230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5\": rpc error: code = NotFound desc = could not find container \"230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5\": container with ID starting with 230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5 not found: ID does not exist"
Jul 02 14:02:58.123610 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.123540    2567 scope.go:117] "RemoveContainer" containerID="a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016"
Jul 02 14:02:58.123774 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:02:58.123759    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016\": container with ID starting with a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016 not found: ID does not exist" containerID="a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016"
Jul 02 14:02:58.123815 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.123777    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016"} err="failed to get container status \"a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016\": rpc error: code = NotFound desc = could not find container \"a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016\": container with ID starting with a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016 not found: ID does not exist"
Jul 02 14:02:58.123815 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.123791    2567 scope.go:117] "RemoveContainer" containerID="230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5"
Jul 02 14:02:58.123997 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.123981    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5"} err="failed to get container status \"230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5\": rpc error: code = NotFound desc = could not find container \"230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5\": container with ID starting with 230d273c53b975198149a8cd6a38465e650ccc4b416cf993387a8cc988de62b5 not found: ID does not exist"
Jul 02 14:02:58.124039 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.123998    2567 scope.go:117] "RemoveContainer" containerID="a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016"
Jul 02 14:02:58.124191 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.124172    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016"} err="failed to get container status \"a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016\": rpc error: code = NotFound desc = could not find container \"a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016\": container with ID starting with a47953f0a492f574691ae91ebd5ed04d7bc565d1d124e9264effd6126d0f0016 not found: ID does not exist"
Jul 02 14:02:58.127932 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.127907    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"]
Jul 02 14:02:58.132133 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.132115    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jsfdr"]
Jul 02 14:02:58.136752 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.136731    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b805e4b6-7c72-4b54-8f2e-5164ac246aec-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:58.139035 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.139018    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b805e4b6-7c72-4b54-8f2e-5164ac246aec-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:58.370058 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.370026    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:02:58.510461 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.510432    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"]
Jul 02 14:02:58.512931 ip-10-0-137-226 kubenswrapper[2567]: W0702 14:02:58.512902    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb805e4b6_7c72_4b54_8f2e_5164ac246aec.slice/crio-e895e4517697b9c7ffd04b34961b1bd05ae77c8f9d901704fd38d795b80501f1 WatchSource:0}: Error finding container e895e4517697b9c7ffd04b34961b1bd05ae77c8f9d901704fd38d795b80501f1: Status 404 returned error can't find the container with id e895e4517697b9c7ffd04b34961b1bd05ae77c8f9d901704fd38d795b80501f1
Jul 02 14:02:58.903920 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:58.903881    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" path="/var/lib/kubelet/pods/92cfa623-e6fb-4676-a9d9-1ea1d08e0778/volumes"
Jul 02 14:02:59.103322 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:59.103294    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:02:59.115763 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:59.115725    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" event={"ID":"b805e4b6-7c72-4b54-8f2e-5164ac246aec","Type":"ContainerStarted","Data":"b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f"}
Jul 02 14:02:59.116125 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:02:59.115777    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" event={"ID":"b805e4b6-7c72-4b54-8f2e-5164ac246aec","Type":"ContainerStarted","Data":"e895e4517697b9c7ffd04b34961b1bd05ae77c8f9d901704fd38d795b80501f1"}
Jul 02 14:03:00.739262 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:00.739217    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 02 14:03:03.133903 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:03.133870    2567 generic.go:356] "Generic (PLEG): container finished" podID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerID="cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921" exitCode=0
Jul 02 14:03:03.134293 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:03.133939    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" event={"ID":"de2c995d-e3fb-4639-bf9e-eae8fceb15e0","Type":"ContainerDied","Data":"cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921"}
Jul 02 14:03:05.738965 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:05.738921    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 02 14:03:05.743784 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:05.743751    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:03:05.744993 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:05.744970    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:03:10.738748 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:10.738663    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 02 14:03:10.739110 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:10.738789    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:03:12.335199 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:12.335166    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:03:15.739204 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:15.739161    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 02 14:03:15.743881 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:15.743849    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:03:15.745110 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:15.745076    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:03:19.202600 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:19.202560    2567 generic.go:356] "Generic (PLEG): container finished" podID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerID="b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f" exitCode=0
Jul 02 14:03:19.202974 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:19.202626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" event={"ID":"b805e4b6-7c72-4b54-8f2e-5164ac246aec","Type":"ContainerDied","Data":"b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f"}
Jul 02 14:03:19.204172 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:19.204151    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 14:03:20.738949 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:20.738902    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 02 14:03:25.738915 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:25.738863    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 02 14:03:25.743880 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:25.743840    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:03:25.744045 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:25.744028    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:03:25.745078 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:25.745038    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 02 14:03:25.745187 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:25.745169    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:03:28.094539 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.094513    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:03:28.169436 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.169399    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-cabundle-cert\") pod \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") "
Jul 02 14:03:28.169649 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.169466    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kserve-provision-location\") pod \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") "
Jul 02 14:03:28.169649 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.169519    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-rbac-proxy-sar-config\") pod \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") "
Jul 02 14:03:28.169649 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.169572    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-proxy-tls\") pod \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") "
Jul 02 14:03:28.169853 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.169825    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "de2c995d-e3fb-4639-bf9e-eae8fceb15e0" (UID: "de2c995d-e3fb-4639-bf9e-eae8fceb15e0"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:03:28.169910 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.169886    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "de2c995d-e3fb-4639-bf9e-eae8fceb15e0" (UID: "de2c995d-e3fb-4639-bf9e-eae8fceb15e0"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:03:28.170018 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.169971    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "de2c995d-e3fb-4639-bf9e-eae8fceb15e0" (UID: "de2c995d-e3fb-4639-bf9e-eae8fceb15e0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:03:28.171840 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.171817    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "de2c995d-e3fb-4639-bf9e-eae8fceb15e0" (UID: "de2c995d-e3fb-4639-bf9e-eae8fceb15e0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:03:28.247772 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.247674    2567 generic.go:356] "Generic (PLEG): container finished" podID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerID="10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34" exitCode=0
Jul 02 14:03:28.247772 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.247745    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"
Jul 02 14:03:28.247772 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.247750    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" event={"ID":"de2c995d-e3fb-4639-bf9e-eae8fceb15e0","Type":"ContainerDied","Data":"10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34"}
Jul 02 14:03:28.248037 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.247837    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m" event={"ID":"de2c995d-e3fb-4639-bf9e-eae8fceb15e0","Type":"ContainerDied","Data":"e8b2e92845731de92d754bd3e9f5d7b361b8019eb834408b2c374b8c6972c552"}
Jul 02 14:03:28.248037 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.247871    2567 scope.go:117] "RemoveContainer" containerID="10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34"
Jul 02 14:03:28.256687 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.256671    2567 scope.go:117] "RemoveContainer" containerID="95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62"
Jul 02 14:03:28.264079 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.264056    2567 scope.go:117] "RemoveContainer" containerID="cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921"
Jul 02 14:03:28.270300 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.270278    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cxqz\" (UniqueName: \"kubernetes.io/projected/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-api-access-2cxqz\") pod \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\" (UID: \"de2c995d-e3fb-4639-bf9e-eae8fceb15e0\") "
Jul 02 14:03:28.270549 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.270531    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-proxy-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:03:28.270600 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.270556    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-cabundle-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:03:28.270600 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.270571    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kserve-provision-location\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:03:28.270600 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.270587    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:03:28.271350 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.271333    2567 scope.go:117] "RemoveContainer" containerID="7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5"
Jul 02 14:03:28.272346 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.272322    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-api-access-2cxqz" (OuterVolumeSpecName: "kube-api-access-2cxqz") pod "de2c995d-e3fb-4639-bf9e-eae8fceb15e0" (UID: "de2c995d-e3fb-4639-bf9e-eae8fceb15e0"). InnerVolumeSpecName "kube-api-access-2cxqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:03:28.281022 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.281003    2567 scope.go:117] "RemoveContainer" containerID="10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34"
Jul 02 14:03:28.281281 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:03:28.281262    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34\": container with ID starting with 10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34 not found: ID does not exist" containerID="10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34"
Jul 02 14:03:28.281349 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.281290    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34"} err="failed to get container status \"10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34\": rpc error: code = NotFound desc = could not find container \"10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34\": container with ID starting with 10e5332574c484c48be7ec84a13b58cec81fa1dfc06194a026ebca134a7c4e34 not found: ID does not exist"
Jul 02 14:03:28.281349 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.281311    2567 scope.go:117] "RemoveContainer" containerID="95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62"
Jul 02 14:03:28.281563 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:03:28.281547    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62\": container with ID starting with 95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62 not found: ID does not exist" containerID="95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62"
Jul 02 14:03:28.281615 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.281570    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62"} err="failed to get container status \"95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62\": rpc error: code = NotFound desc = could not find container \"95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62\": container with ID starting with 95f360c9e2ae1854c034f3b2862c6b6147be333b3345f74c67e7ba0b1eb17f62 not found: ID does not exist"
Jul 02 14:03:28.281615 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.281586    2567 scope.go:117] "RemoveContainer" containerID="cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921"
Jul 02 14:03:28.281837 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:03:28.281821    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921\": container with ID starting with cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921 not found: ID does not exist" containerID="cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921"
Jul 02 14:03:28.281885 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.281841    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921"} err="failed to get container status \"cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921\": rpc error: code = NotFound desc = could not find container \"cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921\": container with ID starting with cebd0b94dd0d2967680a4060e6c51ab92125dc4821178a0e30601ded4638c921 not found: ID does not exist"
Jul 02 14:03:28.281885 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.281854    2567 scope.go:117] "RemoveContainer" containerID="7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5"
Jul 02 14:03:28.282095 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:03:28.282077    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5\": container with ID starting with 7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5 not found: ID does not exist" containerID="7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5"
Jul 02 14:03:28.282136 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.282101    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5"} err="failed to get container status \"7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5\": rpc error: code = NotFound desc = could not find container \"7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5\": container with ID starting with 7044302553e5a93f6e5f02d2426c2d74ce928211df0c125baa6359e91d3cb8d5 not found: ID does not exist"
Jul 02 14:03:28.371029 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.370990    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cxqz\" (UniqueName: \"kubernetes.io/projected/de2c995d-e3fb-4639-bf9e-eae8fceb15e0-kube-api-access-2cxqz\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:03:28.570460 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.570429    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"]
Jul 02 14:03:28.575985 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.575956    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-547b85fd58-gxh5m"]
Jul 02 14:03:28.903095 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:03:28.903020    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" path="/var/lib/kubelet/pods/de2c995d-e3fb-4639-bf9e-eae8fceb15e0/volumes"
Jul 02 14:05:32.828639 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:32.828606    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 14:05:32.829203 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:32.828606    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 14:05:32.836354 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:32.836331    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 14:05:32.836527 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:32.836331    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 14:05:33.779799 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:33.779726    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" event={"ID":"b805e4b6-7c72-4b54-8f2e-5164ac246aec","Type":"ContainerStarted","Data":"785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3"}
Jul 02 14:05:33.779799 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:33.779783    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" event={"ID":"b805e4b6-7c72-4b54-8f2e-5164ac246aec","Type":"ContainerStarted","Data":"01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852"}
Jul 02 14:05:33.780610 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:33.780591    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:05:33.780673 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:33.780645    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:05:33.781776 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:33.781745    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Jul 02 14:05:33.815274 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:33.815223    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" podStartSLOduration=2.3315149330000002 podStartE2EDuration="2m36.815205732s" totalImagesPullingTime="2m14.279128149s" totalInitContainerRuntime="20.20456265s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:02:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:05:33.813384127 +0000 UTC m=+1219.318508652" watchObservedRunningTime="2026-07-02 14:05:33.815205732 +0000 UTC m=+1219.320330236"
Jul 02 14:05:34.783423 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:34.783376    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Jul 02 14:05:35.786847 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:35.786801    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Jul 02 14:05:40.791516 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:40.791471    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:05:40.792041 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:40.792006    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Jul 02 14:05:50.792613 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:50.792581    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:05:57.571263 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:57.571229    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"]
Jul 02 14:05:57.585033 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:57.571472    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kserve-container" containerID="cri-o://01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852" gracePeriod=30
Jul 02 14:05:57.585033 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:57.571521    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kube-rbac-proxy" containerID="cri-o://785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3" gracePeriod=30
Jul 02 14:05:57.646898 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:57.646870    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:05:57.873516 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:57.873414    2567 generic.go:356] "Generic (PLEG): container finished" podID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerID="785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3" exitCode=2
Jul 02 14:05:57.873516 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:57.873475    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" event={"ID":"b805e4b6-7c72-4b54-8f2e-5164ac246aec","Type":"ContainerDied","Data":"785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3"}
Jul 02 14:05:58.446918 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.446888    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:05:58.515373 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.515333    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pl9q\" (UniqueName: \"kubernetes.io/projected/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-api-access-2pl9q\") pod \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") "
Jul 02 14:05:58.515574 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.515392    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-cabundle-cert\") pod \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") "
Jul 02 14:05:58.515574 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.515417    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-rbac-proxy-sar-config\") pod \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") "
Jul 02 14:05:58.515574 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.515440    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kserve-provision-location\") pod \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") "
Jul 02 14:05:58.515710 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.515634    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b805e4b6-7c72-4b54-8f2e-5164ac246aec-proxy-tls\") pod \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\" (UID: \"b805e4b6-7c72-4b54-8f2e-5164ac246aec\") "
Jul 02 14:05:58.515857 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.515827    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "b805e4b6-7c72-4b54-8f2e-5164ac246aec" (UID: "b805e4b6-7c72-4b54-8f2e-5164ac246aec"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:05:58.515857 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.515835    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b805e4b6-7c72-4b54-8f2e-5164ac246aec" (UID: "b805e4b6-7c72-4b54-8f2e-5164ac246aec"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:05:58.517583 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.517511    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-api-access-2pl9q" (OuterVolumeSpecName: "kube-api-access-2pl9q") pod "b805e4b6-7c72-4b54-8f2e-5164ac246aec" (UID: "b805e4b6-7c72-4b54-8f2e-5164ac246aec"). InnerVolumeSpecName "kube-api-access-2pl9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:05:58.517779 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.517754    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b805e4b6-7c72-4b54-8f2e-5164ac246aec-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b805e4b6-7c72-4b54-8f2e-5164ac246aec" (UID: "b805e4b6-7c72-4b54-8f2e-5164ac246aec"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:05:58.518074 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.518054    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b805e4b6-7c72-4b54-8f2e-5164ac246aec" (UID: "b805e4b6-7c72-4b54-8f2e-5164ac246aec"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:05:58.616788 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.616756    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pl9q\" (UniqueName: \"kubernetes.io/projected/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-api-access-2pl9q\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:05:58.616788 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.616783    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-cabundle-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:05:58.616788 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.616795    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:05:58.617242 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.616804    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b805e4b6-7c72-4b54-8f2e-5164ac246aec-kserve-provision-location\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:05:58.617242 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.616812    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b805e4b6-7c72-4b54-8f2e-5164ac246aec-proxy-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:05:58.879340 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.879307    2567 generic.go:356] "Generic (PLEG): container finished" podID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerID="01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852" exitCode=0
Jul 02 14:05:58.879537 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.879369    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"
Jul 02 14:05:58.879537 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.879395    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" event={"ID":"b805e4b6-7c72-4b54-8f2e-5164ac246aec","Type":"ContainerDied","Data":"01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852"}
Jul 02 14:05:58.879537 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.879457    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn" event={"ID":"b805e4b6-7c72-4b54-8f2e-5164ac246aec","Type":"ContainerDied","Data":"e895e4517697b9c7ffd04b34961b1bd05ae77c8f9d901704fd38d795b80501f1"}
Jul 02 14:05:58.879537 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.879502    2567 scope.go:117] "RemoveContainer" containerID="785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3"
Jul 02 14:05:58.889010 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.888987    2567 scope.go:117] "RemoveContainer" containerID="01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852"
Jul 02 14:05:58.902886 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.902864    2567 scope.go:117] "RemoveContainer" containerID="b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f"
Jul 02 14:05:58.905293 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.905268    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"]
Jul 02 14:05:58.910262 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.910238    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-5gppn"]
Jul 02 14:05:58.911927 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.911910    2567 scope.go:117] "RemoveContainer" containerID="785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3"
Jul 02 14:05:58.912223 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:05:58.912206    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3\": container with ID starting with 785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3 not found: ID does not exist" containerID="785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3"
Jul 02 14:05:58.912284 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.912231    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3"} err="failed to get container status \"785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3\": rpc error: code = NotFound desc = could not find container \"785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3\": container with ID starting with 785f0d3b3da168d546cef0498b8b2253154d76f53606c786afd6bdf4276bc4e3 not found: ID does not exist"
Jul 02 14:05:58.912284 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.912248    2567 scope.go:117] "RemoveContainer" containerID="01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852"
Jul 02 14:05:58.912518 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:05:58.912499    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852\": container with ID starting with 01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852 not found: ID does not exist" containerID="01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852"
Jul 02 14:05:58.912580 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.912523    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852"} err="failed to get container status \"01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852\": rpc error: code = NotFound desc = could not find container \"01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852\": container with ID starting with 01475ac9f351d52da7bcf4465271e006f844384fa413a65d478381b5b9669852 not found: ID does not exist"
Jul 02 14:05:58.912580 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.912537    2567 scope.go:117] "RemoveContainer" containerID="b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f"
Jul 02 14:05:58.912772 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:05:58.912752    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f\": container with ID starting with b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f not found: ID does not exist" containerID="b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f"
Jul 02 14:05:58.912819 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:05:58.912779    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f"} err="failed to get container status \"b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f\": rpc error: code = NotFound desc = could not find container \"b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f\": container with ID starting with b741769b2f4186166599001c0fde450ee9840e53e9f6a53268bd4b091319168f not found: ID does not exist"
Jul 02 14:06:00.903127 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:00.903093    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" path="/var/lib/kubelet/pods/b805e4b6-7c72-4b54-8f2e-5164ac246aec/volumes"
Jul 02 14:06:02.864910 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.864879    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:06:02.924469 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924437    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"]
Jul 02 14:06:02.924726 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924714    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kserve-container"
Jul 02 14:06:02.924782 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924728    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kserve-container"
Jul 02 14:06:02.924782 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924738    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="storage-initializer"
Jul 02 14:06:02.924782 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924744    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="storage-initializer"
Jul 02 14:06:02.924782 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924753    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container"
Jul 02 14:06:02.924782 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924759    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container"
Jul 02 14:06:02.924782 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924768    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" containerName="kserve-container"
Jul 02 14:06:02.924782 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924773    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" containerName="kserve-container"
Jul 02 14:06:02.924782 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924779    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="storage-initializer"
Jul 02 14:06:02.924782 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924784    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="storage-initializer"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924792    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kube-rbac-proxy"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924797    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kube-rbac-proxy"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924809    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924814    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924822    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kube-rbac-proxy"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924828    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kube-rbac-proxy"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924833    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" containerName="kube-rbac-proxy"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924838    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" containerName="kube-rbac-proxy"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924882    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" containerName="kserve-container"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924890    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kserve-container"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924896    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="92cfa623-e6fb-4676-a9d9-1ea1d08e0778" containerName="kube-rbac-proxy"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924901    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="kube-rbac-proxy"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924907    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kserve-container"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924913    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="de2c995d-e3fb-4639-bf9e-eae8fceb15e0" containerName="agent"
Jul 02 14:06:02.925086 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.924920    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b805e4b6-7c72-4b54-8f2e-5164ac246aec" containerName="kube-rbac-proxy"
Jul 02 14:06:02.931169 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.931138    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:06:02.958732 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.958698    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"]
Jul 02 14:06:02.958866 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.958824    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:02.961291 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.961270    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 14:06:02.962125 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.962049    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 02 14:06:02.962125 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.962053    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-predictor-serving-cert\""
Jul 02 14:06:02.962125 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.962059    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-dg2z6\""
Jul 02 14:06:02.962125 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.962113    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 02 14:06:02.962413 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.962091    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 14:06:02.962413 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:02.962241    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-kube-rbac-proxy-sar-config\""
Jul 02 14:06:03.052327 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.052286    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/311156e0-cd6a-41d0-a690-71594ce3e37c-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.052327 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.052324    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.052590 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.052355    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4xfz\" (UniqueName: \"kubernetes.io/projected/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-api-access-h4xfz\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.052590 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.052448    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/311156e0-cd6a-41d0-a690-71594ce3e37c-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.052590 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.052506    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.153290 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.153202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/311156e0-cd6a-41d0-a690-71594ce3e37c-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.153290 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.153240    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.153290 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.153279    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/311156e0-cd6a-41d0-a690-71594ce3e37c-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.153585 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.153302    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.153585 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.153337    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4xfz\" (UniqueName: \"kubernetes.io/projected/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-api-access-h4xfz\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.153701 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.153653    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/311156e0-cd6a-41d0-a690-71594ce3e37c-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.153963 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.153939    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.154078 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.154040    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.155653 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.155635    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/311156e0-cd6a-41d0-a690-71594ce3e37c-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.162693 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.162666    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4xfz\" (UniqueName: \"kubernetes.io/projected/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-api-access-h4xfz\") pod \"isvc-autogluon-v2-predictor-67b849d456-cmlfw\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.269715 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.269675    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:03.424186 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.424046    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"]
Jul 02 14:06:03.426914 ip-10-0-137-226 kubenswrapper[2567]: W0702 14:06:03.426890    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod311156e0_cd6a_41d0_a690_71594ce3e37c.slice/crio-a5b952faa6a615b0ffb9fa01467c5b781187c38df1e472ec529b46305d1bbfd4 WatchSource:0}: Error finding container a5b952faa6a615b0ffb9fa01467c5b781187c38df1e472ec529b46305d1bbfd4: Status 404 returned error can't find the container with id a5b952faa6a615b0ffb9fa01467c5b781187c38df1e472ec529b46305d1bbfd4
Jul 02 14:06:03.911025 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.910988    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw" event={"ID":"311156e0-cd6a-41d0-a690-71594ce3e37c","Type":"ContainerStarted","Data":"9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af"}
Jul 02 14:06:03.911411 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:03.911093    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw" event={"ID":"311156e0-cd6a-41d0-a690-71594ce3e37c","Type":"ContainerStarted","Data":"a5b952faa6a615b0ffb9fa01467c5b781187c38df1e472ec529b46305d1bbfd4"}
Jul 02 14:06:08.930745 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:08.930712    2567 generic.go:356] "Generic (PLEG): container finished" podID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerID="9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af" exitCode=0
Jul 02 14:06:08.931246 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:08.930792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw" event={"ID":"311156e0-cd6a-41d0-a690-71594ce3e37c","Type":"ContainerDied","Data":"9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af"}
Jul 02 14:06:09.937112 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:09.937076    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw" event={"ID":"311156e0-cd6a-41d0-a690-71594ce3e37c","Type":"ContainerStarted","Data":"caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce"}
Jul 02 14:06:09.937611 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:09.937149    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw" event={"ID":"311156e0-cd6a-41d0-a690-71594ce3e37c","Type":"ContainerStarted","Data":"5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0"}
Jul 02 14:06:09.937611 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:09.937354    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:09.961001 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:09.960922    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw" podStartSLOduration=3.160435725 podStartE2EDuration="7.960899067s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.800463342s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:06:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:06:09.958901556 +0000 UTC m=+1255.464026056" watchObservedRunningTime="2026-07-02 14:06:09.960899067 +0000 UTC m=+1255.466023572"
Jul 02 14:06:10.940715 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:10.940684    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:16.950340 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:16.950311    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:06:42.084117 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:06:42.084086    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:47.902712 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:47.902630    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:07:53.024419 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:53.024385    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"]
Jul 02 14:07:53.024910 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:53.024640    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw" podUID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerName="kserve-container" containerID="cri-o://5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0" gracePeriod=30
Jul 02 14:07:53.024910 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:53.024719    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw" podUID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerName="kube-rbac-proxy" containerID="cri-o://caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce" gracePeriod=30
Jul 02 14:07:53.051823 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:53.051795    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:53.339847 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:53.339814    2567 generic.go:356] "Generic (PLEG): container finished" podID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerID="caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce" exitCode=2
Jul 02 14:07:53.340029 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:53.339871    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw" event={"ID":"311156e0-cd6a-41d0-a690-71594ce3e37c","Type":"ContainerDied","Data":"caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce"}
Jul 02 14:07:53.972229 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:53.972202    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:07:54.069947 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.069852    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-cabundle-cert\") pod \"311156e0-cd6a-41d0-a690-71594ce3e37c\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") "
Jul 02 14:07:54.069947 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.069914    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-rbac-proxy-sar-config\") pod \"311156e0-cd6a-41d0-a690-71594ce3e37c\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") "
Jul 02 14:07:54.069947 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.069952    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/311156e0-cd6a-41d0-a690-71594ce3e37c-proxy-tls\") pod \"311156e0-cd6a-41d0-a690-71594ce3e37c\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") "
Jul 02 14:07:54.070528 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.069986    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/311156e0-cd6a-41d0-a690-71594ce3e37c-kserve-provision-location\") pod \"311156e0-cd6a-41d0-a690-71594ce3e37c\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") "
Jul 02 14:07:54.070528 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.070134    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4xfz\" (UniqueName: \"kubernetes.io/projected/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-api-access-h4xfz\") pod \"311156e0-cd6a-41d0-a690-71594ce3e37c\" (UID: \"311156e0-cd6a-41d0-a690-71594ce3e37c\") "
Jul 02 14:07:54.070528 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.070330    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "311156e0-cd6a-41d0-a690-71594ce3e37c" (UID: "311156e0-cd6a-41d0-a690-71594ce3e37c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:07:54.070528 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.070399    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "311156e0-cd6a-41d0-a690-71594ce3e37c" (UID: "311156e0-cd6a-41d0-a690-71594ce3e37c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:07:54.072176 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.072150    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311156e0-cd6a-41d0-a690-71594ce3e37c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "311156e0-cd6a-41d0-a690-71594ce3e37c" (UID: "311156e0-cd6a-41d0-a690-71594ce3e37c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:07:54.072278 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.072193    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-api-access-h4xfz" (OuterVolumeSpecName: "kube-api-access-h4xfz") pod "311156e0-cd6a-41d0-a690-71594ce3e37c" (UID: "311156e0-cd6a-41d0-a690-71594ce3e37c"). InnerVolumeSpecName "kube-api-access-h4xfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:07:54.076050 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.076022    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/311156e0-cd6a-41d0-a690-71594ce3e37c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "311156e0-cd6a-41d0-a690-71594ce3e37c" (UID: "311156e0-cd6a-41d0-a690-71594ce3e37c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:07:54.171451 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.171413    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/311156e0-cd6a-41d0-a690-71594ce3e37c-proxy-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:07:54.171451 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.171446    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/311156e0-cd6a-41d0-a690-71594ce3e37c-kserve-provision-location\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:07:54.171451 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.171459    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4xfz\" (UniqueName: \"kubernetes.io/projected/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-api-access-h4xfz\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:07:54.171712 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.171472    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-cabundle-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:07:54.171712 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.171512    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/311156e0-cd6a-41d0-a690-71594ce3e37c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:07:54.346140 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.346052    2567 generic.go:356] "Generic (PLEG): container finished" podID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerID="5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0" exitCode=0
Jul 02 14:07:54.346140 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.346089    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw" event={"ID":"311156e0-cd6a-41d0-a690-71594ce3e37c","Type":"ContainerDied","Data":"5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0"}
Jul 02 14:07:54.346140 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.346110    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"
Jul 02 14:07:54.346419 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.346149    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw" event={"ID":"311156e0-cd6a-41d0-a690-71594ce3e37c","Type":"ContainerDied","Data":"a5b952faa6a615b0ffb9fa01467c5b781187c38df1e472ec529b46305d1bbfd4"}
Jul 02 14:07:54.346419 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.346182    2567 scope.go:117] "RemoveContainer" containerID="caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce"
Jul 02 14:07:54.354982 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.354947    2567 scope.go:117] "RemoveContainer" containerID="5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0"
Jul 02 14:07:54.367208 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.367185    2567 scope.go:117] "RemoveContainer" containerID="9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af"
Jul 02 14:07:54.372338 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.372301    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"]
Jul 02 14:07:54.375291 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.375273    2567 scope.go:117] "RemoveContainer" containerID="caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce"
Jul 02 14:07:54.375632 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:07:54.375587    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce\": container with ID starting with caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce not found: ID does not exist" containerID="caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce"
Jul 02 14:07:54.375734 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.375626    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce"} err="failed to get container status \"caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce\": rpc error: code = NotFound desc = could not find container \"caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce\": container with ID starting with caa9d5e630c616989f067adc7861ff89ed1f0302cb4eac6c6a529afd8632c8ce not found: ID does not exist"
Jul 02 14:07:54.375734 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.375646    2567 scope.go:117] "RemoveContainer" containerID="5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0"
Jul 02 14:07:54.376010 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:07:54.375905    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0\": container with ID starting with 5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0 not found: ID does not exist" containerID="5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0"
Jul 02 14:07:54.376010 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.375939    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0"} err="failed to get container status \"5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0\": rpc error: code = NotFound desc = could not find container \"5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0\": container with ID starting with 5a31123d39481a8c82cdef3885406b8ead770dbcbc0429f2210fcddff79147a0 not found: ID does not exist"
Jul 02 14:07:54.376010 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.375963    2567 scope.go:117] "RemoveContainer" containerID="9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af"
Jul 02 14:07:54.376422 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:07:54.376398    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af\": container with ID starting with 9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af not found: ID does not exist" containerID="9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af"
Jul 02 14:07:54.376531 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.376428    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af"} err="failed to get container status \"9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af\": rpc error: code = NotFound desc = could not find container \"9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af\": container with ID starting with 9be0c2cc19e109e63e4746b96a655a94b09534dd1f4c073a939dfc6fa4b1c1af not found: ID does not exist"
Jul 02 14:07:54.377898 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.377879    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-cmlfw"]
Jul 02 14:07:54.903103 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:54.903071    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="311156e0-cd6a-41d0-a690-71594ce3e37c" path="/var/lib/kubelet/pods/311156e0-cd6a-41d0-a690-71594ce3e37c/volumes"
Jul 02 14:07:58.115040 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.115007    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:58.171338 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.171298    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"]
Jul 02 14:07:58.171700 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.171681    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerName="storage-initializer"
Jul 02 14:07:58.171787 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.171703    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerName="storage-initializer"
Jul 02 14:07:58.171787 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.171715    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerName="kube-rbac-proxy"
Jul 02 14:07:58.171787 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.171724    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerName="kube-rbac-proxy"
Jul 02 14:07:58.171787 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.171759    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerName="kserve-container"
Jul 02 14:07:58.171787 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.171768    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerName="kserve-container"
Jul 02 14:07:58.172051 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.171855    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerName="kserve-container"
Jul 02 14:07:58.172051 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.171870    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="311156e0-cd6a-41d0-a690-71594ce3e37c" containerName="kube-rbac-proxy"
Jul 02 14:07:58.175322 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.175296    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.177834 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.177779    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-predictor-serving-cert\""
Jul 02 14:07:58.177834 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.177821    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 14:07:58.178048 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.177867    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 02 14:07:58.178048 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.177876    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 02 14:07:58.178048 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.177952    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-kube-rbac-proxy-sar-config\""
Jul 02 14:07:58.178165 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.178055    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 14:07:58.178592 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.178573    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-dg2z6\""
Jul 02 14:07:58.185673 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.185645    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"]
Jul 02 14:07:58.203945 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.203904    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.204150 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.203970    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.204150 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.204026    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.204150 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.204057    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll2f4\" (UniqueName: \"kubernetes.io/projected/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-api-access-ll2f4\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.204150 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.204098    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.305106 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.305075    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.305308 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.305118    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.305308 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.305165    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.305308 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.305186    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll2f4\" (UniqueName: \"kubernetes.io/projected/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-api-access-ll2f4\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.305308 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.305212    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.305729 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.305711    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.305891 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.305872    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.305936 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.305889    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.307700 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.307682    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.313974 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.313950    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll2f4\" (UniqueName: \"kubernetes.io/projected/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-api-access-ll2f4\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.332517 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.332451    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:58.487693 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.487589    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:07:58.624247 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:58.624221    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"]
Jul 02 14:07:58.626762 ip-10-0-137-226 kubenswrapper[2567]: W0702 14:07:58.626735    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84f0f02c_b7f1_4126_b36c_76ca98f6f5d3.slice/crio-2bbc499c7511b7b3ece07592dd5aa725ca6d2c5d12778ff2c0948235b4328203 WatchSource:0}: Error finding container 2bbc499c7511b7b3ece07592dd5aa725ca6d2c5d12778ff2c0948235b4328203: Status 404 returned error can't find the container with id 2bbc499c7511b7b3ece07592dd5aa725ca6d2c5d12778ff2c0948235b4328203
Jul 02 14:07:59.367283 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:59.367248    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr" event={"ID":"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3","Type":"ContainerStarted","Data":"6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8"}
Jul 02 14:07:59.367800 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:07:59.367305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr" event={"ID":"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3","Type":"ContainerStarted","Data":"2bbc499c7511b7b3ece07592dd5aa725ca6d2c5d12778ff2c0948235b4328203"}
Jul 02 14:08:27.479976 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:08:27.479938    2567 generic.go:356] "Generic (PLEG): container finished" podID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerID="6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8" exitCode=0
Jul 02 14:08:27.480412 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:08:27.480012    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr" event={"ID":"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3","Type":"ContainerDied","Data":"6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8"}
Jul 02 14:08:28.486604 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:08:28.486565    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr" event={"ID":"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3","Type":"ContainerStarted","Data":"6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58"}
Jul 02 14:08:28.487092 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:08:28.486634    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr" event={"ID":"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3","Type":"ContainerStarted","Data":"8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8"}
Jul 02 14:08:28.487092 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:08:28.486892    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:08:28.509244 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:08:28.509190    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr" podStartSLOduration=1.82515512 podStartE2EDuration="30.509171787s" totalImagesPullingTime="0s" totalInitContainerRuntime="28.684016667s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:07:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:08:28.507661954 +0000 UTC m=+1394.012786459" watchObservedRunningTime="2026-07-02 14:08:28.509171787 +0000 UTC m=+1394.014296291"
Jul 02 14:08:29.490599 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:08:29.490566    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:08:35.500988 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:08:35.500957    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:09:12.531089 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:09:12.531002    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:10:02.903322 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:02.903294    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:10:08.326717 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:08.326684    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"]
Jul 02 14:10:08.327126 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:08.326937    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr" podUID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerName="kserve-container" containerID="cri-o://8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8" gracePeriod=30
Jul 02 14:10:08.327126 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:08.326967    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr" podUID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerName="kube-rbac-proxy" containerID="cri-o://6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58" gracePeriod=30
Jul 02 14:10:08.352627 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:08.352583    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:10:08.875450 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:08.875417    2567 generic.go:356] "Generic (PLEG): container finished" podID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerID="6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58" exitCode=2
Jul 02 14:10:08.875669 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:08.875512    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr" event={"ID":"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3","Type":"ContainerDied","Data":"6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58"}
Jul 02 14:10:09.370516 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.370474    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:10:09.464508 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.464386    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-rbac-proxy-sar-config\") pod \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") "
Jul 02 14:10:09.464508 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.464473    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-cabundle-cert\") pod \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") "
Jul 02 14:10:09.464750 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.464529    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kserve-provision-location\") pod \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") "
Jul 02 14:10:09.464750 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.464584    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-proxy-tls\") pod \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") "
Jul 02 14:10:09.464750 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.464616    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ll2f4\" (UniqueName: \"kubernetes.io/projected/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-api-access-ll2f4\") pod \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\" (UID: \"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3\") "
Jul 02 14:10:09.464929 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.464853    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" (UID: "84f0f02c-b7f1-4126-b36c-76ca98f6f5d3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:10:09.464929 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.464902    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" (UID: "84f0f02c-b7f1-4126-b36c-76ca98f6f5d3"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:10:09.466786 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.466756    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-api-access-ll2f4" (OuterVolumeSpecName: "kube-api-access-ll2f4") pod "84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" (UID: "84f0f02c-b7f1-4126-b36c-76ca98f6f5d3"). InnerVolumeSpecName "kube-api-access-ll2f4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:10:09.466885 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.466829    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" (UID: "84f0f02c-b7f1-4126-b36c-76ca98f6f5d3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:10:09.469214 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.469193    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" (UID: "84f0f02c-b7f1-4126-b36c-76ca98f6f5d3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:10:09.565428 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.565386    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-cabundle-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:10:09.565428 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.565418    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kserve-provision-location\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:10:09.565428 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.565430    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-proxy-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:10:09.565693 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.565441    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ll2f4\" (UniqueName: \"kubernetes.io/projected/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-api-access-ll2f4\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:10:09.565693 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.565450    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:10:09.881372 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.881339    2567 generic.go:356] "Generic (PLEG): container finished" podID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerID="8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8" exitCode=0
Jul 02 14:10:09.881590 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.881395    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"
Jul 02 14:10:09.881590 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.881396    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr" event={"ID":"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3","Type":"ContainerDied","Data":"8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8"}
Jul 02 14:10:09.881590 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.881458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr" event={"ID":"84f0f02c-b7f1-4126-b36c-76ca98f6f5d3","Type":"ContainerDied","Data":"2bbc499c7511b7b3ece07592dd5aa725ca6d2c5d12778ff2c0948235b4328203"}
Jul 02 14:10:09.881590 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.881516    2567 scope.go:117] "RemoveContainer" containerID="6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58"
Jul 02 14:10:09.889980 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.889962    2567 scope.go:117] "RemoveContainer" containerID="8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8"
Jul 02 14:10:09.897362 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.897340    2567 scope.go:117] "RemoveContainer" containerID="6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8"
Jul 02 14:10:09.903523 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.903474    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"]
Jul 02 14:10:09.905915 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.905888    2567 scope.go:117] "RemoveContainer" containerID="6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58"
Jul 02 14:10:09.906387 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:10:09.906357    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58\": container with ID starting with 6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58 not found: ID does not exist" containerID="6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58"
Jul 02 14:10:09.906476 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.906399    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58"} err="failed to get container status \"6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58\": rpc error: code = NotFound desc = could not find container \"6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58\": container with ID starting with 6a24568264365e58e36bec978d228e93f66297685645eb827ced02b570894c58 not found: ID does not exist"
Jul 02 14:10:09.906476 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.906426    2567 scope.go:117] "RemoveContainer" containerID="8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8"
Jul 02 14:10:09.906769 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:10:09.906737    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8\": container with ID starting with 8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8 not found: ID does not exist" containerID="8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8"
Jul 02 14:10:09.906838 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.906765    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8"} err="failed to get container status \"8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8\": rpc error: code = NotFound desc = could not find container \"8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8\": container with ID starting with 8c836a7c18c8ed5d12d9b2f8d6374de3eb6b943f71470fce2d1be4a1b822afe8 not found: ID does not exist"
Jul 02 14:10:09.906838 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.906786    2567 scope.go:117] "RemoveContainer" containerID="6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8"
Jul 02 14:10:09.907045 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:10:09.907030    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8\": container with ID starting with 6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8 not found: ID does not exist" containerID="6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8"
Jul 02 14:10:09.907095 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.907049    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8"} err="failed to get container status \"6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8\": rpc error: code = NotFound desc = could not find container \"6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8\": container with ID starting with 6979e50f4a45699e0b54dbe486082f81218cccf1b738f3eea2d67b94f0346fe8 not found: ID does not exist"
Jul 02 14:10:09.907698 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:09.907677    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-f52lr"]
Jul 02 14:10:10.902805 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:10.902774    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" path="/var/lib/kubelet/pods/84f0f02c-b7f1-4126-b36c-76ca98f6f5d3/volumes"
Jul 02 14:10:13.389136 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.389101    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:10:13.453150 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.453106    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"]
Jul 02 14:10:13.453419 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.453406    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerName="kube-rbac-proxy"
Jul 02 14:10:13.453462 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.453421    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerName="kube-rbac-proxy"
Jul 02 14:10:13.453462 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.453434    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerName="storage-initializer"
Jul 02 14:10:13.453462 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.453440    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerName="storage-initializer"
Jul 02 14:10:13.453462 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.453450    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerName="kserve-container"
Jul 02 14:10:13.453462 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.453456    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerName="kserve-container"
Jul 02 14:10:13.453641 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.453538    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerName="kube-rbac-proxy"
Jul 02 14:10:13.453641 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.453551    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="84f0f02c-b7f1-4126-b36c-76ca98f6f5d3" containerName="kserve-container"
Jul 02 14:10:13.457384 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.457359    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.459983 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.459961    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 02 14:10:13.459983 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.459977    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 14:10:13.460825 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.460803    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-dg2z6\""
Jul 02 14:10:13.461028 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.460979    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 02 14:10:13.461186 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.460908    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 14:10:13.461368 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.460982    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-kube-rbac-proxy-sar-config\""
Jul 02 14:10:13.461438 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.460843    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-predictor-serving-cert\""
Jul 02 14:10:13.469211 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.469185    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"]
Jul 02 14:10:13.593929 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.593890    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.594171 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.593935    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ff35238-bd7b-475f-bcad-894d06ad3089-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.594171 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.593966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ff35238-bd7b-475f-bcad-894d06ad3089-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.594171 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.594063    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7jbx\" (UniqueName: \"kubernetes.io/projected/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-api-access-j7jbx\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.594171 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.594098    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.695606 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.695471    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7jbx\" (UniqueName: \"kubernetes.io/projected/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-api-access-j7jbx\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.695606 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.695549    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.695853 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.695611    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.695853 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.695642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ff35238-bd7b-475f-bcad-894d06ad3089-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.695853 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.695666    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ff35238-bd7b-475f-bcad-894d06ad3089-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.695853 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:10:13.695810    2567 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-serving-cert: secret "isvc-autogluon-v2-noslash-predictor-serving-cert" not found
Jul 02 14:10:13.696056 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:10:13.695874    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff35238-bd7b-475f-bcad-894d06ad3089-proxy-tls podName:8ff35238-bd7b-475f-bcad-894d06ad3089 nodeName:}" failed. No retries permitted until 2026-07-02 14:10:14.195854485 +0000 UTC m=+1499.700978983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/8ff35238-bd7b-475f-bcad-894d06ad3089-proxy-tls") pod "isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" (UID: "8ff35238-bd7b-475f-bcad-894d06ad3089") : secret "isvc-autogluon-v2-noslash-predictor-serving-cert" not found
Jul 02 14:10:13.696120 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.696079    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ff35238-bd7b-475f-bcad-894d06ad3089-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.696412 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.696388    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.696449 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.696415    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.704503 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.704448    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7jbx\" (UniqueName: \"kubernetes.io/projected/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-api-access-j7jbx\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:13.850812 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:13.850765    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:10:14.199627 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:14.199563    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ff35238-bd7b-475f-bcad-894d06ad3089-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:14.201994 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:14.201958    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ff35238-bd7b-475f-bcad-894d06ad3089-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:14.369561 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:14.369523    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:14.509475 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:14.509324    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"]
Jul 02 14:10:14.512302 ip-10-0-137-226 kubenswrapper[2567]: W0702 14:10:14.512275    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff35238_bd7b_475f_bcad_894d06ad3089.slice/crio-a54c5b4a6a8d3c20da86b1b235875bb95b1a9a51abb3d6f6fb06a5c036fa8e6f WatchSource:0}: Error finding container a54c5b4a6a8d3c20da86b1b235875bb95b1a9a51abb3d6f6fb06a5c036fa8e6f: Status 404 returned error can't find the container with id a54c5b4a6a8d3c20da86b1b235875bb95b1a9a51abb3d6f6fb06a5c036fa8e6f
Jul 02 14:10:14.903993 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:14.903953    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" event={"ID":"8ff35238-bd7b-475f-bcad-894d06ad3089","Type":"ContainerStarted","Data":"fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984"}
Jul 02 14:10:14.904147 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:14.904029    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" event={"ID":"8ff35238-bd7b-475f-bcad-894d06ad3089","Type":"ContainerStarted","Data":"a54c5b4a6a8d3c20da86b1b235875bb95b1a9a51abb3d6f6fb06a5c036fa8e6f"}
Jul 02 14:10:19.924947 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:19.924910    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerID="fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984" exitCode=0
Jul 02 14:10:19.925401 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:19.924982    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" event={"ID":"8ff35238-bd7b-475f-bcad-894d06ad3089","Type":"ContainerDied","Data":"fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984"}
Jul 02 14:10:20.932499 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:20.932450    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" event={"ID":"8ff35238-bd7b-475f-bcad-894d06ad3089","Type":"ContainerStarted","Data":"cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c"}
Jul 02 14:10:20.932902 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:20.932535    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" event={"ID":"8ff35238-bd7b-475f-bcad-894d06ad3089","Type":"ContainerStarted","Data":"9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829"}
Jul 02 14:10:20.932902 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:20.932748    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:20.932902 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:20.932818    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:20.962934 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:20.962868    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" podStartSLOduration=3.304169094 podStartE2EDuration="7.962844214s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.65867512s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:10:20.961283911 +0000 UTC m=+1506.466408415" watchObservedRunningTime="2026-07-02 14:10:20.962844214 +0000 UTC m=+1506.467968718"
Jul 02 14:10:26.944506 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:26.944462    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:10:32.858319 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:32.858292    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 14:10:32.858874 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:32.858855    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 14:10:32.866228 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:32.866197    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 14:10:32.866635 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:10:32.866616    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 14:11:52.903674 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:52.903646    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:11:53.584292 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:53.584257    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"]
Jul 02 14:11:54.293737 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:54.293701    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" podUID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerName="kserve-container" containerID="cri-o://9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829" gracePeriod=30
Jul 02 14:11:54.293737 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:54.293726    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" podUID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerName="kube-rbac-proxy" containerID="cri-o://cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c" gracePeriod=30
Jul 02 14:11:54.721615 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:54.721586    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:11:55.243932 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.243908    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:11:55.299063 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.299029    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerID="cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c" exitCode=2
Jul 02 14:11:55.299063 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.299052    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerID="9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829" exitCode=0
Jul 02 14:11:55.299580 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.299095    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"
Jul 02 14:11:55.299580 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.299120    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" event={"ID":"8ff35238-bd7b-475f-bcad-894d06ad3089","Type":"ContainerDied","Data":"cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c"}
Jul 02 14:11:55.299580 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.299188    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" event={"ID":"8ff35238-bd7b-475f-bcad-894d06ad3089","Type":"ContainerDied","Data":"9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829"}
Jul 02 14:11:55.299580 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.299220    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9" event={"ID":"8ff35238-bd7b-475f-bcad-894d06ad3089","Type":"ContainerDied","Data":"a54c5b4a6a8d3c20da86b1b235875bb95b1a9a51abb3d6f6fb06a5c036fa8e6f"}
Jul 02 14:11:55.299580 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.299259    2567 scope.go:117] "RemoveContainer" containerID="cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c"
Jul 02 14:11:55.310366 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.310345    2567 scope.go:117] "RemoveContainer" containerID="9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829"
Jul 02 14:11:55.318277 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.318259    2567 scope.go:117] "RemoveContainer" containerID="fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984"
Jul 02 14:11:55.323357 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.323336    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-cabundle-cert\") pod \"8ff35238-bd7b-475f-bcad-894d06ad3089\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") "
Jul 02 14:11:55.323460 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.323366    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-rbac-proxy-sar-config\") pod \"8ff35238-bd7b-475f-bcad-894d06ad3089\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") "
Jul 02 14:11:55.323460 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.323405    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7jbx\" (UniqueName: \"kubernetes.io/projected/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-api-access-j7jbx\") pod \"8ff35238-bd7b-475f-bcad-894d06ad3089\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") "
Jul 02 14:11:55.323460 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.323425    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ff35238-bd7b-475f-bcad-894d06ad3089-proxy-tls\") pod \"8ff35238-bd7b-475f-bcad-894d06ad3089\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") "
Jul 02 14:11:55.323690 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.323522    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ff35238-bd7b-475f-bcad-894d06ad3089-kserve-provision-location\") pod \"8ff35238-bd7b-475f-bcad-894d06ad3089\" (UID: \"8ff35238-bd7b-475f-bcad-894d06ad3089\") "
Jul 02 14:11:55.323897 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.323870    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "8ff35238-bd7b-475f-bcad-894d06ad3089" (UID: "8ff35238-bd7b-475f-bcad-894d06ad3089"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:11:55.324002 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.323972    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8ff35238-bd7b-475f-bcad-894d06ad3089" (UID: "8ff35238-bd7b-475f-bcad-894d06ad3089"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:11:55.325507 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.325462    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ff35238-bd7b-475f-bcad-894d06ad3089-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8ff35238-bd7b-475f-bcad-894d06ad3089" (UID: "8ff35238-bd7b-475f-bcad-894d06ad3089"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:11:55.325660 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.325644    2567 scope.go:117] "RemoveContainer" containerID="cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c"
Jul 02 14:11:55.325711 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.325670    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-api-access-j7jbx" (OuterVolumeSpecName: "kube-api-access-j7jbx") pod "8ff35238-bd7b-475f-bcad-894d06ad3089" (UID: "8ff35238-bd7b-475f-bcad-894d06ad3089"). InnerVolumeSpecName "kube-api-access-j7jbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:11:55.325968 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:11:55.325948    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c\": container with ID starting with cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c not found: ID does not exist" containerID="cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c"
Jul 02 14:11:55.326015 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.325975    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c"} err="failed to get container status \"cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c\": rpc error: code = NotFound desc = could not find container \"cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c\": container with ID starting with cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c not found: ID does not exist"
Jul 02 14:11:55.326015 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.325993    2567 scope.go:117] "RemoveContainer" containerID="9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829"
Jul 02 14:11:55.326240 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:11:55.326223    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829\": container with ID starting with 9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829 not found: ID does not exist" containerID="9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829"
Jul 02 14:11:55.326307 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.326243    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829"} err="failed to get container status \"9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829\": rpc error: code = NotFound desc = could not find container \"9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829\": container with ID starting with 9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829 not found: ID does not exist"
Jul 02 14:11:55.326307 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.326255    2567 scope.go:117] "RemoveContainer" containerID="fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984"
Jul 02 14:11:55.326536 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:11:55.326508    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984\": container with ID starting with fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984 not found: ID does not exist" containerID="fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984"
Jul 02 14:11:55.326650 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.326542    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984"} err="failed to get container status \"fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984\": rpc error: code = NotFound desc = could not find container \"fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984\": container with ID starting with fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984 not found: ID does not exist"
Jul 02 14:11:55.326650 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.326563    2567 scope.go:117] "RemoveContainer" containerID="cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c"
Jul 02 14:11:55.326818 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.326799    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c"} err="failed to get container status \"cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c\": rpc error: code = NotFound desc = could not find container \"cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c\": container with ID starting with cdc9dc95ad07069ad11f79146f817eb8abfeef7c92be0d096a7cdc3cf85af23c not found: ID does not exist"
Jul 02 14:11:55.326881 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.326818    2567 scope.go:117] "RemoveContainer" containerID="9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829"
Jul 02 14:11:55.327022 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.327003    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829"} err="failed to get container status \"9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829\": rpc error: code = NotFound desc = could not find container \"9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829\": container with ID starting with 9fbbe74b78e76b7ff7d8e7c0f77454f075a8829f590d7c09ebb67449516f0829 not found: ID does not exist"
Jul 02 14:11:55.327088 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.327021    2567 scope.go:117] "RemoveContainer" containerID="fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984"
Jul 02 14:11:55.327227 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.327211    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984"} err="failed to get container status \"fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984\": rpc error: code = NotFound desc = could not find container \"fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984\": container with ID starting with fcbc47ce52e8860a5fed74248745bd1737b5e674858fabeb10f2fed3fd2cf984 not found: ID does not exist"
Jul 02 14:11:55.327974 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.327952    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff35238-bd7b-475f-bcad-894d06ad3089-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8ff35238-bd7b-475f-bcad-894d06ad3089" (UID: "8ff35238-bd7b-475f-bcad-894d06ad3089"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:11:55.424738 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.424648    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ff35238-bd7b-475f-bcad-894d06ad3089-kserve-provision-location\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:11:55.424738 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.424675    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-cabundle-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:11:55.424738 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.424685    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:11:55.424738 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.424697    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7jbx\" (UniqueName: \"kubernetes.io/projected/8ff35238-bd7b-475f-bcad-894d06ad3089-kube-api-access-j7jbx\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:11:55.424738 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.424706    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ff35238-bd7b-475f-bcad-894d06ad3089-proxy-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:11:55.622066 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.622036    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"]
Jul 02 14:11:55.626764 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:55.626734    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-j79k9"]
Jul 02 14:11:56.902625 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:56.902596    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ff35238-bd7b-475f-bcad-894d06ad3089" path="/var/lib/kubelet/pods/8ff35238-bd7b-475f-bcad-894d06ad3089/volumes"
Jul 02 14:11:58.676661 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.676628    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:11:58.765453 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.765414    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"]
Jul 02 14:11:58.766286 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.766260    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerName="kserve-container"
Jul 02 14:11:58.766286 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.766290    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerName="kserve-container"
Jul 02 14:11:58.766502 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.766329    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerName="storage-initializer"
Jul 02 14:11:58.766502 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.766345    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerName="storage-initializer"
Jul 02 14:11:58.766502 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.766364    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerName="kube-rbac-proxy"
Jul 02 14:11:58.766502 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.766374    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerName="kube-rbac-proxy"
Jul 02 14:11:58.766701 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.766516    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerName="kserve-container"
Jul 02 14:11:58.766701 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.766529    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff35238-bd7b-475f-bcad-894d06ad3089" containerName="kube-rbac-proxy"
Jul 02 14:11:58.772337 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.772312    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.775201 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.775176    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 02 14:11:58.775341 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.775205    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-dg2z6\""
Jul 02 14:11:58.775853 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.775833    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 02 14:11:58.775962 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.775864    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-predictor-serving-cert\""
Jul 02 14:11:58.775962 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.775875    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 14:11:58.775962 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.775920    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-kube-rbac-proxy-sar-config\""
Jul 02 14:11:58.775962 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.775870    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 14:11:58.777170 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.777149    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:11:58.782326 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.782306    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"]
Jul 02 14:11:58.854263 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.854222    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8d40d79-4218-400d-ab79-10f7b4f0f367-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.854465 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.854317    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.854465 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.854353    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhfms\" (UniqueName: \"kubernetes.io/projected/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-api-access-nhfms\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.854465 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.854381    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8d40d79-4218-400d-ab79-10f7b4f0f367-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.854465 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.854411    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.954894 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.954798    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.954894 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.954844    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhfms\" (UniqueName: \"kubernetes.io/projected/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-api-access-nhfms\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.954894 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.954881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8d40d79-4218-400d-ab79-10f7b4f0f367-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.955192 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.954920    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.955192 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.954951    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8d40d79-4218-400d-ab79-10f7b4f0f367-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.955400 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.955367    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8d40d79-4218-400d-ab79-10f7b4f0f367-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.955644 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.955625    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.955716 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.955654    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.957312 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.957292    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8d40d79-4218-400d-ab79-10f7b4f0f367-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:58.964639 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:58.964611    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhfms\" (UniqueName: \"kubernetes.io/projected/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-api-access-nhfms\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:59.083678 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:59.083634    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:11:59.212984 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:59.212961    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"]
Jul 02 14:11:59.215656 ip-10-0-137-226 kubenswrapper[2567]: W0702 14:11:59.215629    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d40d79_4218_400d_ab79_10f7b4f0f367.slice/crio-515c2b30360c6394e6ea53cdd1ed5a9303ce6773052c13fe21306696edf9110d WatchSource:0}: Error finding container 515c2b30360c6394e6ea53cdd1ed5a9303ce6773052c13fe21306696edf9110d: Status 404 returned error can't find the container with id 515c2b30360c6394e6ea53cdd1ed5a9303ce6773052c13fe21306696edf9110d
Jul 02 14:11:59.318729 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:59.318693    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" event={"ID":"b8d40d79-4218-400d-ab79-10f7b4f0f367","Type":"ContainerStarted","Data":"3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b"}
Jul 02 14:11:59.318899 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:11:59.318750    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" event={"ID":"b8d40d79-4218-400d-ab79-10f7b4f0f367","Type":"ContainerStarted","Data":"515c2b30360c6394e6ea53cdd1ed5a9303ce6773052c13fe21306696edf9110d"}
Jul 02 14:12:08.355782 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:08.355745    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerID="3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b" exitCode=0
Jul 02 14:12:08.356267 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:08.355799    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" event={"ID":"b8d40d79-4218-400d-ab79-10f7b4f0f367","Type":"ContainerDied","Data":"3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b"}
Jul 02 14:12:09.362255 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:09.362214    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" event={"ID":"b8d40d79-4218-400d-ab79-10f7b4f0f367","Type":"ContainerStarted","Data":"52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3"}
Jul 02 14:12:09.362667 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:09.362335    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" event={"ID":"b8d40d79-4218-400d-ab79-10f7b4f0f367","Type":"ContainerStarted","Data":"963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad"}
Jul 02 14:12:09.362667 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:09.362591    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:12:09.385102 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:09.385044    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" podStartSLOduration=2.637640704 podStartE2EDuration="11.385026503s" totalImagesPullingTime="0s" totalInitContainerRuntime="8.747385799s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:11:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:12:09.384260169 +0000 UTC m=+1614.889384683" watchObservedRunningTime="2026-07-02 14:12:09.385026503 +0000 UTC m=+1614.890151006"
Jul 02 14:12:10.366431 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:10.366396    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:12:10.367786 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:10.367755    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 02 14:12:11.371350 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:11.371315    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 02 14:12:16.375703 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:16.375672    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:12:16.376169 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:16.376141    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 02 14:12:26.377344 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:26.377307    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:12:28.804866 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:28.804834    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"]
Jul 02 14:12:28.805285 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:28.805046    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="kserve-container" containerID="cri-o://963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad" gracePeriod=30
Jul 02 14:12:28.805285 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:28.805114    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="kube-rbac-proxy" containerID="cri-o://52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3" gracePeriod=30
Jul 02 14:12:28.877687 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:28.877654    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:12:29.448290 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.448256    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerID="52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3" exitCode=2
Jul 02 14:12:29.448465 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.448325    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" event={"ID":"b8d40d79-4218-400d-ab79-10f7b4f0f367","Type":"ContainerDied","Data":"52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3"}
Jul 02 14:12:29.756211 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.756186    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:12:29.804405 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.804374    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-cabundle-cert\") pod \"b8d40d79-4218-400d-ab79-10f7b4f0f367\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") "
Jul 02 14:12:29.804405 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.804414    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-rbac-proxy-sar-config\") pod \"b8d40d79-4218-400d-ab79-10f7b4f0f367\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") "
Jul 02 14:12:29.804684 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.804519    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8d40d79-4218-400d-ab79-10f7b4f0f367-proxy-tls\") pod \"b8d40d79-4218-400d-ab79-10f7b4f0f367\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") "
Jul 02 14:12:29.804684 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.804588    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8d40d79-4218-400d-ab79-10f7b4f0f367-kserve-provision-location\") pod \"b8d40d79-4218-400d-ab79-10f7b4f0f367\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") "
Jul 02 14:12:29.804684 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.804630    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhfms\" (UniqueName: \"kubernetes.io/projected/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-api-access-nhfms\") pod \"b8d40d79-4218-400d-ab79-10f7b4f0f367\" (UID: \"b8d40d79-4218-400d-ab79-10f7b4f0f367\") "
Jul 02 14:12:29.804895 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.804875    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b8d40d79-4218-400d-ab79-10f7b4f0f367" (UID: "b8d40d79-4218-400d-ab79-10f7b4f0f367"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:12:29.804895 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.804880    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "b8d40d79-4218-400d-ab79-10f7b4f0f367" (UID: "b8d40d79-4218-400d-ab79-10f7b4f0f367"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:12:29.805462 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.805442    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8d40d79-4218-400d-ab79-10f7b4f0f367-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b8d40d79-4218-400d-ab79-10f7b4f0f367" (UID: "b8d40d79-4218-400d-ab79-10f7b4f0f367"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:12:29.806673 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.806654    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8d40d79-4218-400d-ab79-10f7b4f0f367-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b8d40d79-4218-400d-ab79-10f7b4f0f367" (UID: "b8d40d79-4218-400d-ab79-10f7b4f0f367"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:12:29.806736 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.806664    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-api-access-nhfms" (OuterVolumeSpecName: "kube-api-access-nhfms") pod "b8d40d79-4218-400d-ab79-10f7b4f0f367" (UID: "b8d40d79-4218-400d-ab79-10f7b4f0f367"). InnerVolumeSpecName "kube-api-access-nhfms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:12:29.905310 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.905269    2567 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8d40d79-4218-400d-ab79-10f7b4f0f367-kserve-provision-location\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:12:29.905310 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.905303    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhfms\" (UniqueName: \"kubernetes.io/projected/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-api-access-nhfms\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:12:29.905310 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.905314    2567 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-cabundle-cert\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:12:29.905602 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.905323    2567 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b8d40d79-4218-400d-ab79-10f7b4f0f367-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:12:29.905602 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:29.905332    2567 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8d40d79-4218-400d-ab79-10f7b4f0f367-proxy-tls\") on node \"ip-10-0-137-226.ec2.internal\" DevicePath \"\""
Jul 02 14:12:30.453850 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.453813    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerID="963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad" exitCode=0
Jul 02 14:12:30.454018 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.453868    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"
Jul 02 14:12:30.454018 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.453874    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" event={"ID":"b8d40d79-4218-400d-ab79-10f7b4f0f367","Type":"ContainerDied","Data":"963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad"}
Jul 02 14:12:30.454018 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.453929    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6" event={"ID":"b8d40d79-4218-400d-ab79-10f7b4f0f367","Type":"ContainerDied","Data":"515c2b30360c6394e6ea53cdd1ed5a9303ce6773052c13fe21306696edf9110d"}
Jul 02 14:12:30.454018 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.453955    2567 scope.go:117] "RemoveContainer" containerID="52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3"
Jul 02 14:12:30.462150 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.462129    2567 scope.go:117] "RemoveContainer" containerID="963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad"
Jul 02 14:12:30.469571 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.469551    2567 scope.go:117] "RemoveContainer" containerID="3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b"
Jul 02 14:12:30.476887 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.476861    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"]
Jul 02 14:12:30.477162 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.477146    2567 scope.go:117] "RemoveContainer" containerID="52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3"
Jul 02 14:12:30.477449 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:12:30.477429    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3\": container with ID starting with 52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3 not found: ID does not exist" containerID="52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3"
Jul 02 14:12:30.477543 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.477457    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3"} err="failed to get container status \"52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3\": rpc error: code = NotFound desc = could not find container \"52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3\": container with ID starting with 52bb2f383e50eb4153fd36a3abc06e8b485e9d8464910ca918a73c49881892d3 not found: ID does not exist"
Jul 02 14:12:30.477543 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.477491    2567 scope.go:117] "RemoveContainer" containerID="963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad"
Jul 02 14:12:30.477726 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:12:30.477710    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad\": container with ID starting with 963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad not found: ID does not exist" containerID="963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad"
Jul 02 14:12:30.477761 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.477729    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad"} err="failed to get container status \"963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad\": rpc error: code = NotFound desc = could not find container \"963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad\": container with ID starting with 963dacae2b55c4043d0e9915309d1229d38926c7562173dfe29216a72e1384ad not found: ID does not exist"
Jul 02 14:12:30.477761 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.477742    2567 scope.go:117] "RemoveContainer" containerID="3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b"
Jul 02 14:12:30.477935 ip-10-0-137-226 kubenswrapper[2567]: E0702 14:12:30.477916    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b\": container with ID starting with 3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b not found: ID does not exist" containerID="3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b"
Jul 02 14:12:30.477971 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.477942    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b"} err="failed to get container status \"3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b\": rpc error: code = NotFound desc = could not find container \"3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b\": container with ID starting with 3f378322a9bf2aa2b8bc09e760f00ba8815776c953d170208851b504e3d8ea6b not found: ID does not exist"
Jul 02 14:12:30.481230 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.481214    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hdtp6"]
Jul 02 14:12:30.902318 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:30.902286    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" path="/var/lib/kubelet/pods/b8d40d79-4218-400d-ab79-10f7b4f0f367/volumes"
Jul 02 14:12:35.142163 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:35.142134    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:12:35.233751 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:12:35.233719    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:13:03.500745 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.500705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:13:03.620384 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.620353    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:13:03.763840 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.763756    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hs76s/must-gather-7h2s5"]
Jul 02 14:13:03.764035 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.764024    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="kserve-container"
Jul 02 14:13:03.764082 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.764037    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="kserve-container"
Jul 02 14:13:03.764082 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.764057    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="storage-initializer"
Jul 02 14:13:03.764082 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.764063    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="storage-initializer"
Jul 02 14:13:03.764082 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.764071    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="kube-rbac-proxy"
Jul 02 14:13:03.764082 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.764077    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="kube-rbac-proxy"
Jul 02 14:13:03.764238 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.764121    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="kube-rbac-proxy"
Jul 02 14:13:03.764238 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.764131    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8d40d79-4218-400d-ab79-10f7b4f0f367" containerName="kserve-container"
Jul 02 14:13:03.767078 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.767063    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hs76s/must-gather-7h2s5"
Jul 02 14:13:03.769624 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.769593    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hs76s\"/\"kube-root-ca.crt\""
Jul 02 14:13:03.769768 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.769709    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hs76s\"/\"openshift-service-ca.crt\""
Jul 02 14:13:03.769833 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.769781    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hs76s\"/\"default-dockercfg-2gmp6\""
Jul 02 14:13:03.777447 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.777418    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hs76s/must-gather-7h2s5"]
Jul 02 14:13:03.836326 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.836288    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:13:03.879422 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.879393    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ef47f1e8-6650-46b3-b4e7-6aeec8878e64-must-gather-output\") pod \"must-gather-7h2s5\" (UID: \"ef47f1e8-6650-46b3-b4e7-6aeec8878e64\") " pod="openshift-must-gather-hs76s/must-gather-7h2s5"
Jul 02 14:13:03.879605 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.879436    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d24gl\" (UniqueName: \"kubernetes.io/projected/ef47f1e8-6650-46b3-b4e7-6aeec8878e64-kube-api-access-d24gl\") pod \"must-gather-7h2s5\" (UID: \"ef47f1e8-6650-46b3-b4e7-6aeec8878e64\") " pod="openshift-must-gather-hs76s/must-gather-7h2s5"
Jul 02 14:13:03.980734 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.980699    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ef47f1e8-6650-46b3-b4e7-6aeec8878e64-must-gather-output\") pod \"must-gather-7h2s5\" (UID: \"ef47f1e8-6650-46b3-b4e7-6aeec8878e64\") " pod="openshift-must-gather-hs76s/must-gather-7h2s5"
Jul 02 14:13:03.980734 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.980742    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d24gl\" (UniqueName: \"kubernetes.io/projected/ef47f1e8-6650-46b3-b4e7-6aeec8878e64-kube-api-access-d24gl\") pod \"must-gather-7h2s5\" (UID: \"ef47f1e8-6650-46b3-b4e7-6aeec8878e64\") " pod="openshift-must-gather-hs76s/must-gather-7h2s5"
Jul 02 14:13:03.981070 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.981049    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ef47f1e8-6650-46b3-b4e7-6aeec8878e64-must-gather-output\") pod \"must-gather-7h2s5\" (UID: \"ef47f1e8-6650-46b3-b4e7-6aeec8878e64\") " pod="openshift-must-gather-hs76s/must-gather-7h2s5"
Jul 02 14:13:03.989962 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:03.989940    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d24gl\" (UniqueName: \"kubernetes.io/projected/ef47f1e8-6650-46b3-b4e7-6aeec8878e64-kube-api-access-d24gl\") pod \"must-gather-7h2s5\" (UID: \"ef47f1e8-6650-46b3-b4e7-6aeec8878e64\") " pod="openshift-must-gather-hs76s/must-gather-7h2s5"
Jul 02 14:13:04.075794 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:04.075752    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hs76s/must-gather-7h2s5"
Jul 02 14:13:04.205250 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:04.205181    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hs76s/must-gather-7h2s5"]
Jul 02 14:13:04.207812 ip-10-0-137-226 kubenswrapper[2567]: W0702 14:13:04.207785    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef47f1e8_6650_46b3_b4e7_6aeec8878e64.slice/crio-806f7e1c68db07d3162fd2e952c357f5a238a70f01f06841eeff8f130cd7e7a3 WatchSource:0}: Error finding container 806f7e1c68db07d3162fd2e952c357f5a238a70f01f06841eeff8f130cd7e7a3: Status 404 returned error can't find the container with id 806f7e1c68db07d3162fd2e952c357f5a238a70f01f06841eeff8f130cd7e7a3
Jul 02 14:13:04.209905 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:04.209886    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 14:13:04.583389 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:04.583356    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hs76s/must-gather-7h2s5" event={"ID":"ef47f1e8-6650-46b3-b4e7-6aeec8878e64","Type":"ContainerStarted","Data":"806f7e1c68db07d3162fd2e952c357f5a238a70f01f06841eeff8f130cd7e7a3"}
Jul 02 14:13:05.591199 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:05.590130    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hs76s/must-gather-7h2s5" event={"ID":"ef47f1e8-6650-46b3-b4e7-6aeec8878e64","Type":"ContainerStarted","Data":"1cf1eeb6afa31ee105f3dfb79927874069a863b92b78fb8e1f70b0d992374cd4"}
Jul 02 14:13:06.596109 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:06.596073    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hs76s/must-gather-7h2s5" event={"ID":"ef47f1e8-6650-46b3-b4e7-6aeec8878e64","Type":"ContainerStarted","Data":"cd6ae664ce1b87115f5f7bf7cbd181a5228266d4d720aefc4ad008c1b91c804b"}
Jul 02 14:13:06.613074 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:06.613009    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hs76s/must-gather-7h2s5" podStartSLOduration=2.347872914 podStartE2EDuration="3.612986714s" totalImagesPullingTime="1.2651138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:13:06.611741281 +0000 UTC m=+1672.116865785" watchObservedRunningTime="2026-07-02 14:13:06.612986714 +0000 UTC m=+1672.118111220"
Jul 02 14:13:07.017358 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:07.017267    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jbfz7_49ea922e-112c-480c-9a70-7992f0e29149/global-pull-secret-syncer/0.log"
Jul 02 14:13:07.164853 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:07.164820    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log"
Jul 02 14:13:07.355217 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:07.355182    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-226.ec2.internal_f18bf94e33493ecb15787b6bbd7e24d0/haproxy/0.log"
Jul 02 14:13:09.200387 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:09.200313    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:13:09.303137 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:09.303052    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sdgz_a44edce6-3bb4-47ec-ac53-7724912c6277/konnectivity-agent/0.log" limit=1024
Jul 02 14:13:11.688025 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:11.687996    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-zwkbz_219d776d-ee12-4b78-a891-e26ffd6d8b41/kube-state-metrics/0.log"
Jul 02 14:13:11.719044 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:11.718990    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-zwkbz_219d776d-ee12-4b78-a891-e26ffd6d8b41/kube-rbac-proxy-main/0.log"
Jul 02 14:13:11.745044 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:11.745013    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-zwkbz_219d776d-ee12-4b78-a891-e26ffd6d8b41/kube-rbac-proxy-self/0.log"
Jul 02 14:13:12.048522 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:12.048470    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vdkwc_40e2287b-c0f8-49ef-b444-da316078e626/node-exporter/0.log"
Jul 02 14:13:12.085900 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:12.085856    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vdkwc_40e2287b-c0f8-49ef-b444-da316078e626/kube-rbac-proxy/0.log"
Jul 02 14:13:12.112790 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:12.112754    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vdkwc_40e2287b-c0f8-49ef-b444-da316078e626/init-textfile/0.log"
Jul 02 14:13:12.437668 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:12.437586    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-dtx5r_0c4da376-ce5b-4cfd-a22d-aacd9920f2fe/prometheus-operator/0.log"
Jul 02 14:13:12.463824 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:12.463785    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-dtx5r_0c4da376-ce5b-4cfd-a22d-aacd9920f2fe/kube-rbac-proxy/0.log"
Jul 02 14:13:14.530137 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:14.530105    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/1.log"
Jul 02 14:13:14.537447 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:14.537417    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-jjmnl_fc379a40-405c-417d-bb40-ab9064a2b14b/console-operator/2.log"
Jul 02 14:13:14.988414 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:14.988385    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5499b6fb8f-fnjd9_1ae29235-cca2-4a37-80ae-65fef194d3fc/download-server/0.log"
Jul 02 14:13:15.713176 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.713136    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"]
Jul 02 14:13:15.718639 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.718612    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.726575 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.725518    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"]
Jul 02 14:13:15.895245 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.895206    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww7l4\" (UniqueName: \"kubernetes.io/projected/63e2484a-bf72-4f9c-b6d6-401e907425a3-kube-api-access-ww7l4\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.895426 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.895263    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-sys\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.895426 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.895287    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-lib-modules\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.895426 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.895332    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-podres\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.895426 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.895396    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-proc\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.995867 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.995784    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-sys\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.995867 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.995817    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-lib-modules\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.995867 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.995858    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-podres\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.996149 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.995903    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-sys\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.996149 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.995918    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-proc\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.996149 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.995962    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww7l4\" (UniqueName: \"kubernetes.io/projected/63e2484a-bf72-4f9c-b6d6-401e907425a3-kube-api-access-ww7l4\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.996149 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.995989    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-lib-modules\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.996149 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.996042    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-proc\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:15.996149 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:15.996051    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63e2484a-bf72-4f9c-b6d6-401e907425a3-podres\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:16.004742 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:16.004711    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww7l4\" (UniqueName: \"kubernetes.io/projected/63e2484a-bf72-4f9c-b6d6-401e907425a3-kube-api-access-ww7l4\") pod \"perf-node-gather-daemonset-68k6v\" (UID: \"63e2484a-bf72-4f9c-b6d6-401e907425a3\") " pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:16.032216 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:16.032171    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:16.224765 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:16.224730    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"]
Jul 02 14:13:16.607720 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:16.607663    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9h5lq_b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6/dns/0.log"
Jul 02 14:13:16.631454 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:16.631428    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9h5lq_b253b67b-e1fc-4f4b-a20d-25cd3d07c7f6/kube-rbac-proxy/0.log"
Jul 02 14:13:16.651561 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:16.651524    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v" event={"ID":"63e2484a-bf72-4f9c-b6d6-401e907425a3","Type":"ContainerStarted","Data":"926f365957c2c5715ac29e73c27755cfa1c9a1642c47e86113a124bec048ecbb"}
Jul 02 14:13:16.651755 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:16.651586    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v" event={"ID":"63e2484a-bf72-4f9c-b6d6-401e907425a3","Type":"ContainerStarted","Data":"a32f6a3de6815b8677c3adceaafe8530b27b041ed67487dc63b0749c9e0cc030"}
Jul 02 14:13:16.652038 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:16.652012    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:16.672788 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:16.672733    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v" podStartSLOduration=1.67271709 podStartE2EDuration="1.67271709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:13:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:13:16.670793689 +0000 UTC m=+1682.175918185" watchObservedRunningTime="2026-07-02 14:13:16.67271709 +0000 UTC m=+1682.177841593"
Jul 02 14:13:16.776092 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:16.776059    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zqz2f_747984cc-ab52-45dc-99ef-53c93e5bd101/dns-node-resolver/0.log"
Jul 02 14:13:17.375471 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:17.375434    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-644d865c6b-mk8jh_585ed721-7d68-462d-9431-17135ff5475b/registry/0.log"
Jul 02 14:13:17.383513 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:17.383453    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-644d865c6b-mk8jh_585ed721-7d68-462d-9431-17135ff5475b/registry/1.log"
Jul 02 14:13:17.434556 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:17.434529    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f8rvf_110466f9-e7b0-4a07-8a64-126c9ebaf9c1/node-ca/0.log"
Jul 02 14:13:18.320856 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:18.320833    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c987749f-vpfgn_c3b71b52-1d0a-4b4e-98d7-6a1e9fcfdac7/router/0.log"
Jul 02 14:13:18.670780 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:18.670705    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kfvfm_99d73dda-2da1-400e-bc32-a373ba20f546/serve-healthcheck-canary/0.log"
Jul 02 14:13:19.110629 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:19.110598    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-z4sj4_75095516-19ab-4843-aeb8-8b4df9fc1005/insights-operator/0.log"
Jul 02 14:13:19.110629 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:19.110631    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-z4sj4_75095516-19ab-4843-aeb8-8b4df9fc1005/insights-operator/1.log"
Jul 02 14:13:19.287696 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:19.287648    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nxgcc_d753af15-bfcf-4f37-8d5e-d894521e6e5b/kube-rbac-proxy/0.log"
Jul 02 14:13:19.315734 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:19.315703    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nxgcc_d753af15-bfcf-4f37-8d5e-d894521e6e5b/exporter/0.log"
Jul 02 14:13:19.339973 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:19.339944    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nxgcc_d753af15-bfcf-4f37-8d5e-d894521e6e5b/extractor/0.log"
Jul 02 14:13:21.536678 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:21.536641    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-7c747fbc6-hrc9l_69470978-4c8f-455c-9f12-154c6a6f64fd/manager/0.log"
Jul 02 14:13:21.559403 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:21.559361    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-67d95fdc48-p9npl_5e189a31-5139-4291-bc2d-90f85ad58384/manager/0.log"
Jul 02 14:13:22.667474 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:22.667449    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hs76s/perf-node-gather-daemonset-68k6v"
Jul 02 14:13:28.050655 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:28.050619    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n88db_f24875be-dc41-4b03-84e6-e00f7c5d9df0/kube-multus-additional-cni-plugins/0.log"
Jul 02 14:13:28.078191 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:28.078164    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n88db_f24875be-dc41-4b03-84e6-e00f7c5d9df0/egress-router-binary-copy/0.log"
Jul 02 14:13:28.107105 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:28.107079    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n88db_f24875be-dc41-4b03-84e6-e00f7c5d9df0/cni-plugins/0.log"
Jul 02 14:13:28.131117 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:28.131086    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n88db_f24875be-dc41-4b03-84e6-e00f7c5d9df0/bond-cni-plugin/0.log"
Jul 02 14:13:28.154227 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:28.154201    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n88db_f24875be-dc41-4b03-84e6-e00f7c5d9df0/routeoverride-cni/0.log"
Jul 02 14:13:28.178841 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:28.178816    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n88db_f24875be-dc41-4b03-84e6-e00f7c5d9df0/whereabouts-cni-bincopy/0.log"
Jul 02 14:13:28.241328 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:28.241292    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n88db_f24875be-dc41-4b03-84e6-e00f7c5d9df0/whereabouts-cni/0.log"
Jul 02 14:13:28.507820 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:28.507721    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-md66t_247c1841-374e-4648-b211-aa3a08b28f06/kube-multus/0.log"
Jul 02 14:13:28.640766 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:28.640719    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-whrk8_d01be1dd-2bb1-409d-aaaa-6094d6e05f26/network-metrics-daemon/0.log"
Jul 02 14:13:28.665864 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:28.665837    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-whrk8_d01be1dd-2bb1-409d-aaaa-6094d6e05f26/kube-rbac-proxy/0.log"
Jul 02 14:13:30.017475 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:30.017448    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-controller/0.log"
Jul 02 14:13:30.038840 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:30.038808    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/0.log"
Jul 02 14:13:30.039051 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:30.038815    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovn-acl-logging/1.log"
Jul 02 14:13:30.066396 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:30.066371    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/kube-rbac-proxy-node/0.log"
Jul 02 14:13:30.090545 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:30.090516    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/kube-rbac-proxy-ovn-metrics/0.log"
Jul 02 14:13:30.114302 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:30.114275    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/northd/0.log"
Jul 02 14:13:30.137511 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:30.137468    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/nbdb/0.log"
Jul 02 14:13:30.166158 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:30.166117    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/sbdb/0.log"
Jul 02 14:13:30.232596 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:30.232566    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xtms2_2539e33c-8342-4e24-879a-690d229ac2e9/ovnkube-controller/0.log"
Jul 02 14:13:31.340853 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:31.340824    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-bt4j2_afddd3e6-b476-40e8-95d8-2b2332236c91/network-check-target-container/0.log"
Jul 02 14:13:32.531912 ip-10-0-137-226 kubenswrapper[2567]: I0702 14:13:32.531884    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-qzt8s_41ed71d6-1988-4c22-b3af-36711ebeef44/iptables-alerter/0.log"