Jul 18 00:40:28.867152 ip-10-0-135-148 systemd[1]: Starting Kubernetes Kubelet... Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.242394 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246316 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246328 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246331 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246336 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246339 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246342 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246345 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 00:40:29.286603 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246348 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246351 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246354 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246356 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246359 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246362 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246364 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246367 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246370 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246373 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246375 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246379 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246381 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246385 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246387 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246390 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246393 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246396 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246399 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246401 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 00:40:29.290625 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246404 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246407 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246410 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246412 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246415 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246418 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246420 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246423 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246426 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246428 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246431 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246437 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246440 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246443 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246445 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246448 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246451 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246454 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246456 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246459 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 00:40:29.293315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246462 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246465 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246468 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246471 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246474 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246476 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246479 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246481 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246485 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246488 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246490 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246493 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246497 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246501 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246504 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246507 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246510 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246513 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246516 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246519 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 00:40:29.306320 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246522 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246526 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246529 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246532 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246541 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246546 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246551 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246553 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246557 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246561 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246563 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246566 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246569 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246571 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246574 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246577 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246579 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246582 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246584 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 00:40:29.322668 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246587 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 00:40:29.314228 ip-10-0-135-148 systemd[1]: Started Kubernetes Kubelet. Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246589 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246592 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246595 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246597 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246599 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246602 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246606 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246609 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246611 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246614 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246616 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.246619 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247383 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247390 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247392 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247397 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247402 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247405 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 00:40:29.331868 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247409 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247412 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247416 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247419 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247421 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247425 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247428 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247431 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247433 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247436 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247438 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247440 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247444 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247447 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247450 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247452 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247455 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247460 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247462 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 00:40:29.347391 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247465 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247468 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247470 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247473 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247476 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247478 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247481 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247483 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247486 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247489 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247491 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247494 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247496 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247499 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247502 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247505 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247508 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247510 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247513 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247516 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 00:40:29.554131 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247519 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247521 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247526 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247528 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247531 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247534 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247536 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247539 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247541 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247544 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247549 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247552 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247555 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247557 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247561 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247565 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247567 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247570 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247572 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247575 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 00:40:29.568177 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247578 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247580 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247584 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247586 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247589 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247591 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247595 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247597 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247600 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247603 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247605 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247608 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247610 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247613 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247616 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247619 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247624 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247627 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247630 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 00:40:29.574662 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247633 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247635 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247638 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247642 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247645 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247647 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247650 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247653 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247655 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247658 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247660 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247663 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247665 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247668 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.247671 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248109 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248115 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248118 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248122 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248125 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 00:40:29.575336 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248129 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248132 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248135 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248138 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248141 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248143 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248147 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248152 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248155 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248158 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248161 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248163 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248166 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248169 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248171 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248176 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248178 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248181 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248185 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 00:40:29.575958 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248189 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248191 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248194 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248197 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248199 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248202 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248204 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248207 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248209 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248212 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248214 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248217 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248219 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248224 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248227 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248230 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248233 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248236 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248238 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 00:40:29.576568 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248241 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248245 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248248 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248251 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248254 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248257 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248260 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248262 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248265 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248269 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248272 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248274 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248277 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248279 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248282 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248285 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248288 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248290 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248293 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248295 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 00:40:29.577152 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248298 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248300 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248303 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248305 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248308 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248310 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248313 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248322 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248325 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248328 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248330 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248332 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248335 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248338 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248341 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248344 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248347 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248349 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248352 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248355 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 00:40:29.577805 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248357 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248362 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248365 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248368 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248371 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248373 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248376 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248378 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248381 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248384 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248386 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248389 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248391 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248394 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248396 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.248399 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248471 2585 flags.go:64] FLAG: --address="0.0.0.0" Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248479 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248485 2585 flags.go:64] FLAG: --anonymous-auth="true" Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248490 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248495 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 18 00:40:29.578402 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248499 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248503 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248507 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248510 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248515 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248519 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248522 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248525 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248528 2585 flags.go:64] FLAG: --cgroup-root="" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248532 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248535 2585 flags.go:64] FLAG: --client-ca-file="" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248538 2585 flags.go:64] FLAG: --cloud-provider="external" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248542 2585 flags.go:64] FLAG: --cluster-dns="[]" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248545 2585 flags.go:64] FLAG: --cluster-domain="" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248548 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248551 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248555 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248558 2585 flags.go:64] FLAG: --container-log-max-files="5" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248562 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248565 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248569 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248572 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248576 2585 flags.go:64] FLAG: --contention-profiling="false" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248579 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 18 00:40:29.579078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248582 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248585 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248588 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248592 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248595 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248598 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248601 2585 flags.go:64] FLAG: --enable-load-reader="false" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248604 2585 flags.go:64] FLAG: --enable-server="true" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248607 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248612 2585 flags.go:64] FLAG: --event-burst="100" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248615 2585 flags.go:64] FLAG: --event-qps="50" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248618 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248622 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248625 2585 flags.go:64] FLAG: --eviction-hard="" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248629 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248632 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248635 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248638 2585 flags.go:64] FLAG: --eviction-soft="" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248640 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248643 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248648 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248651 2585 flags.go:64] FLAG: --experimental-mounter-path="" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248654 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248657 2585 flags.go:64] FLAG: --fail-swap-on="true" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248660 2585 flags.go:64] FLAG: --feature-gates="" Jul 18 00:40:29.579727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248664 2585 flags.go:64] FLAG: --file-check-frequency="20s" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248667 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248670 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248673 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248676 2585 flags.go:64] FLAG: --healthz-port="10248" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248679 2585 flags.go:64] FLAG: --help="false" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248682 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248685 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248688 2585 flags.go:64] FLAG: --http-check-frequency="20s" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248691 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248694 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248697 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248700 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248703 2585 flags.go:64] FLAG: --image-service-endpoint="" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248706 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248709 2585 flags.go:64] FLAG: --kube-api-burst="100" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248712 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248716 2585 flags.go:64] FLAG: --kube-api-qps="50" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248718 2585 flags.go:64] FLAG: --kube-reserved="" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248723 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248726 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248729 2585 flags.go:64] FLAG: --kubelet-cgroups="" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248731 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248734 2585 flags.go:64] FLAG: --lock-file="" Jul 18 00:40:29.580580 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248737 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248740 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248743 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248750 2585 flags.go:64] FLAG: --log-json-split-stream="false" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248753 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248756 2585 flags.go:64] FLAG: --log-text-split-stream="false" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248759 2585 flags.go:64] FLAG: --logging-format="text" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248761 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248764 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248767 2585 flags.go:64] FLAG: --manifest-url="" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248770 2585 flags.go:64] FLAG: --manifest-url-header="" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248775 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248778 2585 flags.go:64] FLAG: --max-open-files="1000000" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248782 2585 flags.go:64] FLAG: --max-pods="110" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248785 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248788 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248791 2585 flags.go:64] FLAG: --memory-manager-policy="None" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248794 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248797 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248800 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248803 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248810 2585 flags.go:64] FLAG: --node-status-max-images="50" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248813 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248815 2585 flags.go:64] FLAG: --oom-score-adj="-999" Jul 18 00:40:29.581243 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248819 2585 flags.go:64] FLAG: --pod-cidr="" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248822 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248845 2585 flags.go:64] FLAG: --pod-manifest-path="" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248848 2585 flags.go:64] FLAG: --pod-max-pids="-1" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248852 2585 flags.go:64] FLAG: --pods-per-core="0" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248855 2585 flags.go:64] FLAG: --port="10250" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248858 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248861 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-090a96da8b64d76b2" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248864 2585 flags.go:64] FLAG: --qos-reserved="" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248867 2585 flags.go:64] FLAG: --read-only-port="10255" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248870 2585 flags.go:64] FLAG: --register-node="true" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248875 2585 flags.go:64] FLAG: --register-with-taints="" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248879 2585 flags.go:64] FLAG: --registry-burst="10" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248881 2585 flags.go:64] FLAG: --registry-qps="5" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248884 2585 flags.go:64] FLAG: --reserved-cpus="" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248887 2585 flags.go:64] FLAG: --reserved-memory="" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248891 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248894 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248897 2585 flags.go:64] FLAG: --rotate-certificates="false" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248900 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248903 2585 flags.go:64] FLAG: --runonce="false" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248906 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248909 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248912 2585 flags.go:64] FLAG: --seccomp-default="false" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248914 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 18 00:40:29.582166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248917 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248920 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248923 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248926 2585 flags.go:64] FLAG: --storage-driver-password="root" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248929 2585 flags.go:64] FLAG: --storage-driver-secure="false" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248932 2585 flags.go:64] FLAG: --storage-driver-table="stats" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248935 2585 flags.go:64] FLAG: --storage-driver-user="root" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248938 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248940 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248944 2585 flags.go:64] FLAG: --system-cgroups="" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248947 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248953 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248956 2585 flags.go:64] FLAG: --tls-cert-file="" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248959 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248962 2585 flags.go:64] FLAG: --tls-min-version="" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248965 2585 flags.go:64] FLAG: --tls-private-key-file="" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248968 2585 flags.go:64] FLAG: --topology-manager-policy="none" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248971 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248976 2585 flags.go:64] FLAG: --topology-manager-scope="container" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248979 2585 flags.go:64] FLAG: --v="2" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248983 2585 flags.go:64] FLAG: --version="false" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248987 2585 flags.go:64] FLAG: --vmodule="" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248991 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.248994 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249112 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 00:40:29.583222 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249116 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249120 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249123 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249126 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249128 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249131 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249134 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249136 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249140 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249143 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249147 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249150 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249152 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249156 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249158 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249161 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249164 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249167 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249172 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 00:40:29.584064 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249175 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249177 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249180 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249182 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249185 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249187 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249191 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249194 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249196 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249199 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249202 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249204 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249207 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249209 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249212 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249214 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249217 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249220 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249222 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249224 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 00:40:29.584677 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249227 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249230 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249232 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249235 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249237 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249240 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249242 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249245 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249248 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249250 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249253 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249257 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249260 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249262 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249265 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249267 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249270 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249272 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249276 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 00:40:29.585518 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249279 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249282 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249285 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249287 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249290 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249292 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249295 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249298 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249300 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249303 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249305 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249308 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249310 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249313 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249315 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249318 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249321 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249323 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249326 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249328 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 00:40:29.586141 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249331 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249333 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249336 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249338 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249342 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249345 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249348 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249350 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249353 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249356 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249358 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249363 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249365 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249368 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249372 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249376 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249378 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249381 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249384 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 00:40:29.586775 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.249388 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.249934 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.259113 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.259129 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.259149 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.259156 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259215 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259220 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259223 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259227 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259230 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259233 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259236 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259239 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259242 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 00:40:29.587401 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259244 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259247 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259250 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259252 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259255 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259258 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259261 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259263 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259266 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259269 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259272 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259275 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259278 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259280 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259283 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259287 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259291 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259295 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259298 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 00:40:29.588028 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259301 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259304 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259307 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259310 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259313 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259315 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259318 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259321 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259324 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259327 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259330 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259333 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259335 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259338 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259341 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259343 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259346 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259349 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259352 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 00:40:29.731276 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259355 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259357 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259360 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259362 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259365 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259368 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259370 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259373 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259375 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259378 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259380 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259382 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259385 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259388 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259390 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259393 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259395 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259398 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259401 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259403 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 00:40:29.731993 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259408 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259410 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259413 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259416 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259419 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259421 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259424 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259426 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259430 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259432 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259435 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259438 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259441 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259445 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259447 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259451 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259454 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259457 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259460 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 00:40:29.733296 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259463 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259465 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259469 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259472 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259475 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259477 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259482 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259485 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259488 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259490 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259493 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259496 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259499 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.259505 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259631 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 00:40:29.733871 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259636 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259639 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259642 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259645 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259648 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259651 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259654 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259656 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259659 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259661 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259664 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259668 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259672 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259676 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259679 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259683 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259686 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259689 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259692 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 00:40:29.734301 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259695 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259697 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259700 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259703 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259705 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259709 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259712 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259714 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259717 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259720 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259722 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259726 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259729 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259732 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259734 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259737 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259739 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259742 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259745 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259747 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 00:40:29.880398 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259750 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259752 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259755 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259758 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259760 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259764 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259766 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259769 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259771 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259774 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259777 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259780 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259782 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259785 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259787 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259790 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259792 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259797 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259799 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259803 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 00:40:29.881070 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259806 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259809 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259812 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259815 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259818 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259821 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259824 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259840 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259843 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259846 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259853 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259856 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259858 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259861 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259864 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259866 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259869 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259871 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259875 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259878 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 00:40:29.881679 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259880 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259883 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259885 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259888 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259891 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259893 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259896 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259898 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259901 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259906 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259908 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259911 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259914 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259917 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259920 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259924 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259926 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259929 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 00:40:29.882298 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:29.259932 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.259937 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.260072 2585 server.go:962] "Client rotation is on, will bootstrap in background" Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.263414 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.264641 2585 server.go:1019] "Starting client certificate rotation" Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.264744 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.264778 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.267019 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.268437 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.280073 2585 log.go:25] "Validated CRI v1 runtime API" Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.282256 2585 log.go:25] "Validated CRI v1 image API" Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.283085 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 18 00:40:29.882820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.285236 2585 fs.go:135] Filesystem UUIDs: map[0088c6df-26fc-4203-a168-8e1e4403c2d2:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b5f2f9cf-d7e9-400e-acb8-728685de1dd7:/dev/nvme0n1p3] Jul 18 00:40:29.883188 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.285252 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 18 00:40:29.883297 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.293489 2585 manager.go:216] Machine: {Timestamp:2026-07-18 00:40:29.291557841 +0000 UTC m=+0.313692619 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105161 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec231489fcde4a552df462ab8acc40fc SystemUUID:ec231489-fcde-4a55-2df4-62ab8acc40fc BootID:33c0bb31-4d03-453e-aa68-8dc27ab94897 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4f:c8:16:57:ab Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4f:c8:16:57:ab Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:9d:b7:40:da:fc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 18 00:40:29.883297 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.293677 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 18 00:40:29.883297 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.293719 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 00:40:29.883297 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.293791 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 18 00:40:29.883297 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.297523 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 18 00:40:29.883297 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.297548 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-148.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 18 00:40:29.883297 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.298222 2585 topology_manager.go:138] "Creating topology manager with none policy" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.298233 2585 container_manager_linux.go:309] "Creating device plugin manager" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.298240 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.298968 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.300121 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.302950 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.303061 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.303105 2585 kubelet.go:486] "Attempting to sync node with API server" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.303114 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.303128 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.303137 2585 kubelet.go:398] "Adding apiserver pod source" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.303146 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.303483 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.303497 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.305475 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.307799 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.307842 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308877 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308891 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308897 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308903 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308912 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308918 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308927 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308932 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308939 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308947 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308956 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.308962 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.310326 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.310337 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.312628 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-148.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.313147 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.313143 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.313402 2585 server.go:1268] "Started kubelet" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.313479 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.313506 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.313537 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.314936 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 18 00:40:29.883546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.315119 2585 server.go:310] "Adding debug handlers to kubelet server" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.320178 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.320434 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.320434 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.320546 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321002 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321003 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321030 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321097 2585 reconstruct.go:97] "Volume reconstruction finished" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321108 2585 reconciler.go:29] "Reconciler: start to sync state" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321219 2585 factory.go:55] Registering systemd factory Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321259 2585 factory.go:223] Registration of the systemd container factory successfully Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321495 2585 factory.go:153] Registering CRI-O factory Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321508 2585 factory.go:223] Registration of the crio container factory successfully Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321556 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321578 2585 factory.go:103] Registering Raw factory Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.321591 2585 manager.go:1196] Started watching for new ooms in manager Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.321672 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-148.ec2.internal.18c33b6fef705427 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-148.ec2.internal,UID:ip-10-0-135-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-148.ec2.internal,},FirstTimestamp:2026-07-18 00:40:29.313373223 +0000 UTC m=+0.335507999,LastTimestamp:2026-07-18 00:40:29.313373223 +0000 UTC m=+0.335507999,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-148.ec2.internal,}" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.323146 2585 manager.go:349] Starting recovery of all containers Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.328364 2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.328444 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.335854 2585 manager.go:354] Recovery completed Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.340355 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.342326 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.342368 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.342381 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.342930 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.342940 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.342949 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.342957 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.343657 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.343685 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 00:40:29.884624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.343695 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.344170 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jw55d" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.344268 2585 policy_none.go:49] "None policy: Start" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.344279 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.344293 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.344348 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-148.ec2.internal.18c33b6ff12a5e0a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-148.ec2.internal,UID:ip-10-0-135-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-148.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-148.ec2.internal,},FirstTimestamp:2026-07-18 00:40:29.342342666 +0000 UTC m=+0.364477442,LastTimestamp:2026-07-18 00:40:29.342342666 +0000 UTC m=+0.364477442,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-148.ec2.internal,}" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.345405 2585 policy_none.go:47] "Start" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.351458 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jw55d" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.354614 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-148.ec2.internal.18c33b6ff12ada45 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-148.ec2.internal,UID:ip-10-0-135-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-148.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-148.ec2.internal,},FirstTimestamp:2026-07-18 00:40:29.342374469 +0000 UTC m=+0.364509244,LastTimestamp:2026-07-18 00:40:29.342374469 +0000 UTC m=+0.364509244,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-148.ec2.internal,}" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.402539 2585 manager.go:337] "Starting Device Plugin manager" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.402585 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.402595 2585 server.go:87] "Starting device plugin registration server" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.402929 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.402942 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.403045 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.403118 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.403126 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.403646 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.403699 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-148.ec2.internal\" not found" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.449055 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.450328 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.450367 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.450393 2585 kubelet.go:2448] "Starting kubelet main sync loop" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.450427 2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.452757 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.503410 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.504471 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory" Jul 18 00:40:29.885547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.504500 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.504510 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.504542 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.515221 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.515245 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-148.ec2.internal\": node \"ip-10-0-135-148.ec2.internal\" not found" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.540887 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.551190 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal"] Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.551277 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.552476 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.552503 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.552516 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.554002 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.554549 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.554585 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.555051 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.555077 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.555092 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.555624 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.555651 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.555665 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.557164 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.557194 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.558065 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientMemory" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.558083 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.558094 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeHasSufficientPID" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.594871 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-148.ec2.internal\" not found" node="ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.600935 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-148.ec2.internal\" not found" node="ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.622586 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fae8633b8e71ee520faef372c752a6b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"fae8633b8e71ee520faef372c752a6b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.886320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.622632 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae8633b8e71ee520faef372c752a6b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"fae8633b8e71ee520faef372c752a6b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.887082 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.622652 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee0ba5c47453590857789475d8c354a6-config\") pod \"kube-apiserver-proxy-ip-10-0-135-148.ec2.internal\" (UID: \"ee0ba5c47453590857789475d8c354a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.887082 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.641507 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found" Jul 18 00:40:29.887082 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.722785 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee0ba5c47453590857789475d8c354a6-config\") pod \"kube-apiserver-proxy-ip-10-0-135-148.ec2.internal\" (UID: \"ee0ba5c47453590857789475d8c354a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.887082 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.722820 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fae8633b8e71ee520faef372c752a6b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"fae8633b8e71ee520faef372c752a6b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.887082 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.722865 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae8633b8e71ee520faef372c752a6b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"fae8633b8e71ee520faef372c752a6b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.887082 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.722874 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee0ba5c47453590857789475d8c354a6-config\") pod \"kube-apiserver-proxy-ip-10-0-135-148.ec2.internal\" (UID: \"ee0ba5c47453590857789475d8c354a6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.887082 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.722916 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae8633b8e71ee520faef372c752a6b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"fae8633b8e71ee520faef372c752a6b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.887082 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.722917 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fae8633b8e71ee520faef372c752a6b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal\" (UID: \"fae8633b8e71ee520faef372c752a6b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" Jul 18 00:40:29.887082 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.741871 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found" Jul 18 00:40:29.887082 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.842699 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found" Jul 18 00:40:30.015958 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.897933 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" Jul 18 00:40:30.015958 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:29.903691 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" Jul 18 00:40:30.015958 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:29.943731 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found" Jul 18 00:40:30.167476 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.044279 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found" Jul 18 00:40:30.167476 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.144763 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-148.ec2.internal\" not found" Jul 18 00:40:30.167476 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.161534 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 00:40:30.245202 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.208921 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 00:40:30.245202 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.220607 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" Jul 18 00:40:30.250302 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.250145 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.251201 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.267439 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.267548 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.267558 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.267549 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.303469 2585 apiserver.go:52] "Watching apiserver" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.311573 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.313113 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-vhhcd","kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal","openshift-cluster-node-tuning-operator/tuned-njl82","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal","openshift-multus/multus-additional-cni-plugins-p729n","openshift-network-diagnostics/network-check-target-hdxtn","openshift-network-operator/iptables-alerter-xxx5d","openshift-ovn-kubernetes/ovnkube-node-c6t5d","kube-system/konnectivity-agent-bl9f9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78","openshift-dns/node-resolver-mfx87","openshift-image-registry/node-ca-cxmc9","openshift-multus/multus-fft7p","openshift-multus/network-metrics-daemon-2zfkq"] Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.315777 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.318738 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.319196 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.319512 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.322366 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tkjgn\"" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.323063 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.324388 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.324959 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.325019 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.325331 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.325684 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xmvdr\"" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.325864 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.326126 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.326319 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.326544 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.326862 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.337931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.327496 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xmj6x\"" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.327680 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.327896 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328679 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frbfw\" (UniqueName: \"kubernetes.io/projected/a8f576bc-9ad2-47cd-a879-43ac2e334a95-kube-api-access-frbfw\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328706 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-multus-cni-dir\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328729 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-sysconfig\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328751 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-run\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328776 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-lib-modules\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328797 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db82a0d6-cda4-453c-8659-f3915518575a-etc-tuned\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328824 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328847 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-registration-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328870 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e18494be-47e6-488a-ae19-adbb9fe95263-cni-binary-copy\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328925 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-var-lib-kubelet\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328952 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-socket-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328973 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.339312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.328994 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-sys-fs\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329016 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-os-release\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329036 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-hostroot\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329058 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-multus-conf-dir\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329084 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e18494be-47e6-488a-ae19-adbb9fe95263-multus-daemon-config\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329108 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-run-multus-certs\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329134 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-host\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329167 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db82a0d6-cda4-453c-8659-f3915518575a-tmp\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329189 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-multus-socket-dir-parent\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329212 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbjxx\" (UniqueName: \"kubernetes.io/projected/e18494be-47e6-488a-ae19-adbb9fe95263-kube-api-access-jbjxx\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329236 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-systemd\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329258 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-var-lib-kubelet\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329284 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzrbm\" (UniqueName: \"kubernetes.io/projected/db82a0d6-cda4-453c-8659-f3915518575a-kube-api-access-jzrbm\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329307 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-device-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.340244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329329 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-etc-kubernetes\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329361 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwlmm\" (UniqueName: \"kubernetes.io/projected/634714a5-7e1e-4a53-97c7-244117cae3cd-kube-api-access-bwlmm\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329383 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-system-cni-dir\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329408 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-cnibin\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329442 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-var-lib-cni-bin\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329474 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-kubernetes\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329497 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-sysctl-d\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329520 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329545 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-run-k8s-cni-cncf-io\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329568 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-run-netns\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329589 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-modprobe-d\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329610 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-sysctl-conf\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329631 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-sys\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.329652 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-var-lib-cni-multus\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.330256 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.330459 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l8hpx\"" Jul 18 00:40:30.340712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.330621 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.330864 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.330931 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.331180 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.333436 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xxx5d" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.334205 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.334438 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7vh6m\"" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.334603 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.335557 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.335884 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.337068 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.337253 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6sfkd\"" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.337418 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.338134 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.338185 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.339733 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.340186 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 18 00:40:30.341414 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.340368 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.352569 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mfx87" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.352584 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8466p\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.352632 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.352900 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.353010 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.354971 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 00:35:30 +0000 UTC" deadline="2028-04-21 13:11:13.058221086 +0000 UTC" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.355018 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15444h30m42.70320777s" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.355796 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cxmc9" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.356268 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.356313 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.356601 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.356686 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6ll72\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.358366 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.358755 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.358813 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7t2bp\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.358911 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.369406 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-66tm5" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.379687 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-66tm5" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.422606 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430171 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-kubernetes\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-run-k8s-cni-cncf-io\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430214 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-run-netns\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430257 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d74ea046-64dd-4ef8-be97-f099264e9890-host-slash\") pod \"iptables-alerter-xxx5d\" (UID: \"d74ea046-64dd-4ef8-be97-f099264e9890\") " pod="openshift-network-operator/iptables-alerter-xxx5d" Jul 18 00:40:30.457428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430282 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55cbdaea-d9a9-4eca-b1c0-65de273699dd-dbus\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430303 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-run-k8s-cni-cncf-io\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430312 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-run-netns\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430311 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430348 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-modprobe-d\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430305 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-kubernetes\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430368 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-var-lib-cni-multus\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430398 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-var-lib-cni-multus\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430415 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c50bca7-78a9-43b3-87a3-74812b70f11b-ovnkube-config\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430436 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-cni-binary-copy\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430453 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-modprobe-d\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430460 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-node-log\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430489 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430506 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d74ea046-64dd-4ef8-be97-f099264e9890-iptables-alerter-script\") pod \"iptables-alerter-xxx5d\" (UID: \"d74ea046-64dd-4ef8-be97-f099264e9890\") " pod="openshift-network-operator/iptables-alerter-xxx5d" Jul 18 00:40:30.459454 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430524 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-sysconfig\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430541 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-lib-modules\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430564 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db82a0d6-cda4-453c-8659-f3915518575a-etc-tuned\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430596 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-sysconfig\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430623 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-registration-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430644 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-lib-modules\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430666 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-etc-kubernetes\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430686 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-registration-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430698 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-kubelet\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430733 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-etc-kubernetes\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430734 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-etc-openvswitch\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430777 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x9xv\" (UniqueName: \"kubernetes.io/projected/1c50bca7-78a9-43b3-87a3-74812b70f11b-kube-api-access-9x9xv\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430797 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430819 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-hostroot\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.460434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430838 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-multus-conf-dir\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430885 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430897 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e18494be-47e6-488a-ae19-adbb9fe95263-multus-daemon-config\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430899 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-hostroot\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbjxx\" (UniqueName: \"kubernetes.io/projected/e18494be-47e6-488a-ae19-adbb9fe95263-kube-api-access-jbjxx\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430935 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-multus-conf-dir\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430951 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-run-ovn\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430973 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.430992 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-host\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431024 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c50bca7-78a9-43b3-87a3-74812b70f11b-env-overrides\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431052 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-host\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431088 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlgj6\" (UniqueName: \"kubernetes.io/projected/d800e91f-1436-4859-83e2-9bd813856736-kube-api-access-rlgj6\") pod \"node-ca-cxmc9\" (UID: \"d800e91f-1436-4859-83e2-9bd813856736\") " pod="openshift-image-registry/node-ca-cxmc9" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431117 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-systemd\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431143 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-var-lib-kubelet\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431150 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-systemd\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.461383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431165 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzrbm\" (UniqueName: \"kubernetes.io/projected/db82a0d6-cda4-453c-8659-f3915518575a-kube-api-access-jzrbm\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431181 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-device-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431185 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-var-lib-kubelet\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431225 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-device-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431200 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-os-release\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431261 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-cnibin\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431285 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-log-socket\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431308 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-sysctl-d\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431333 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431354 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-slash\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431376 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-cnibin\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431382 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-cnibin\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431402 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55cbdaea-d9a9-4eca-b1c0-65de273699dd-kubelet-config\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431429 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-sysctl-conf\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431457 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-sys\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.462049 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.431457 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431533 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-sys\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431485 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-run-netns\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431481 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-sysctl-d\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.431563 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs podName:a8f576bc-9ad2-47cd-a879-43ac2e334a95 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:30.931539935 +0000 UTC m=+1.953674717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs") pod "network-metrics-daemon-2zfkq" (UID: "a8f576bc-9ad2-47cd-a879-43ac2e334a95") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431579 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-etc-sysctl-conf\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431598 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c50bca7-78a9-43b3-87a3-74812b70f11b-ovnkube-script-lib\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431629 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frbfw\" (UniqueName: \"kubernetes.io/projected/a8f576bc-9ad2-47cd-a879-43ac2e334a95-kube-api-access-frbfw\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431657 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-multus-cni-dir\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431683 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-run-ovn-kubernetes\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431710 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431763 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-run\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431789 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.462626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431813 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e18494be-47e6-488a-ae19-adbb9fe95263-cni-binary-copy\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431838 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-var-lib-kubelet\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431874 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db82a0d6-cda4-453c-8659-f3915518575a-run\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431901 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-var-lib-kubelet\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431908 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-run-multus-certs\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431903 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431929 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4ba3c30d-3d5a-463b-918b-555fe7292755-agent-certs\") pod \"konnectivity-agent-bl9f9\" (UID: \"4ba3c30d-3d5a-463b-918b-555fe7292755\") " pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431924 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-multus-cni-dir\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431949 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-var-lib-openvswitch\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431952 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-run-multus-certs\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.431969 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-cni-netd\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432000 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-socket-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432036 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-sys-fs\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432065 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-os-release\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.463091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432074 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-sys-fs\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432086 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-system-cni-dir\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432109 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srzxs\" (UniqueName: \"kubernetes.io/projected/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-kube-api-access-srzxs\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432123 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-os-release\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432150 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l72tx\" (UniqueName: \"kubernetes.io/projected/d74ea046-64dd-4ef8-be97-f099264e9890-kube-api-access-l72tx\") pod \"iptables-alerter-xxx5d\" (UID: \"d74ea046-64dd-4ef8-be97-f099264e9890\") " pod="openshift-network-operator/iptables-alerter-xxx5d" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432168 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3-tmp-dir\") pod \"node-resolver-mfx87\" (UID: \"4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3\") " pod="openshift-dns/node-resolver-mfx87" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432183 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d800e91f-1436-4859-83e2-9bd813856736-serviceca\") pod \"node-ca-cxmc9\" (UID: \"d800e91f-1436-4859-83e2-9bd813856736\") " pod="openshift-image-registry/node-ca-cxmc9" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432209 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db82a0d6-cda4-453c-8659-f3915518575a-tmp\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432230 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-multus-socket-dir-parent\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432246 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432263 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7wmg\" (UniqueName: \"kubernetes.io/projected/4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3-kube-api-access-m7wmg\") pod \"node-resolver-mfx87\" (UID: \"4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3\") " pod="openshift-dns/node-resolver-mfx87" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432285 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-cni-bin\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432310 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx9j9\" (UniqueName: \"kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9\") pod \"network-check-target-hdxtn\" (UID: \"01540c57-73db-4206-bdd9-d83fbcbb8066\") " pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432331 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-multus-socket-dir-parent\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.463764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432334 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3-hosts-file\") pod \"node-resolver-mfx87\" (UID: \"4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3\") " pod="openshift-dns/node-resolver-mfx87" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432380 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d800e91f-1436-4859-83e2-9bd813856736-host\") pod \"node-ca-cxmc9\" (UID: \"d800e91f-1436-4859-83e2-9bd813856736\") " pod="openshift-image-registry/node-ca-cxmc9" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432426 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwlmm\" (UniqueName: \"kubernetes.io/projected/634714a5-7e1e-4a53-97c7-244117cae3cd-kube-api-access-bwlmm\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432463 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-system-cni-dir\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432518 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-var-lib-cni-bin\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432553 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-system-cni-dir\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432581 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4ba3c30d-3d5a-463b-918b-555fe7292755-konnectivity-ca\") pod \"konnectivity-agent-bl9f9\" (UID: \"4ba3c30d-3d5a-463b-918b-555fe7292755\") " pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432604 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-systemd-units\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432635 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e18494be-47e6-488a-ae19-adbb9fe95263-host-var-lib-cni-bin\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432659 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-run-systemd\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432677 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-run-openvswitch\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432705 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c50bca7-78a9-43b3-87a3-74812b70f11b-ovn-node-metrics-cert\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.432870 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/634714a5-7e1e-4a53-97c7-244117cae3cd-socket-dir\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.433124 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.434164 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e18494be-47e6-488a-ae19-adbb9fe95263-cni-binary-copy\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.464282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.434211 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e18494be-47e6-488a-ae19-adbb9fe95263-multus-daemon-config\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.464836 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.436414 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db82a0d6-cda4-453c-8659-f3915518575a-tmp\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.464836 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.438495 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db82a0d6-cda4-453c-8659-f3915518575a-etc-tuned\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.464836 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.460559 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frbfw\" (UniqueName: \"kubernetes.io/projected/a8f576bc-9ad2-47cd-a879-43ac2e334a95-kube-api-access-frbfw\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:30.464836 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.460599 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzrbm\" (UniqueName: \"kubernetes.io/projected/db82a0d6-cda4-453c-8659-f3915518575a-kube-api-access-jzrbm\") pod \"tuned-njl82\" (UID: \"db82a0d6-cda4-453c-8659-f3915518575a\") " pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.464836 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.460599 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbjxx\" (UniqueName: \"kubernetes.io/projected/e18494be-47e6-488a-ae19-adbb9fe95263-kube-api-access-jbjxx\") pod \"multus-fft7p\" (UID: \"e18494be-47e6-488a-ae19-adbb9fe95263\") " pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.464836 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.460808 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwlmm\" (UniqueName: \"kubernetes.io/projected/634714a5-7e1e-4a53-97c7-244117cae3cd-kube-api-access-bwlmm\") pod \"aws-ebs-csi-driver-node-rjp78\" (UID: \"634714a5-7e1e-4a53-97c7-244117cae3cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.532997 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3-tmp-dir\") pod \"node-resolver-mfx87\" (UID: \"4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3\") " pod="openshift-dns/node-resolver-mfx87" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533027 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d800e91f-1436-4859-83e2-9bd813856736-serviceca\") pod \"node-ca-cxmc9\" (UID: \"d800e91f-1436-4859-83e2-9bd813856736\") " pod="openshift-image-registry/node-ca-cxmc9" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533048 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533065 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7wmg\" (UniqueName: \"kubernetes.io/projected/4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3-kube-api-access-m7wmg\") pod \"node-resolver-mfx87\" (UID: \"4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3\") " pod="openshift-dns/node-resolver-mfx87" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533083 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-cni-bin\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533099 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9j9\" (UniqueName: \"kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9\") pod \"network-check-target-hdxtn\" (UID: \"01540c57-73db-4206-bdd9-d83fbcbb8066\") " pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533116 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3-hosts-file\") pod \"node-resolver-mfx87\" (UID: \"4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3\") " pod="openshift-dns/node-resolver-mfx87" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533131 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d800e91f-1436-4859-83e2-9bd813856736-host\") pod \"node-ca-cxmc9\" (UID: \"d800e91f-1436-4859-83e2-9bd813856736\") " pod="openshift-image-registry/node-ca-cxmc9" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533149 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4ba3c30d-3d5a-463b-918b-555fe7292755-konnectivity-ca\") pod \"konnectivity-agent-bl9f9\" (UID: \"4ba3c30d-3d5a-463b-918b-555fe7292755\") " pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533166 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-systemd-units\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533185 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-run-systemd\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533208 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-run-openvswitch\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533273 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c50bca7-78a9-43b3-87a3-74812b70f11b-ovn-node-metrics-cert\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.564715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533305 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d74ea046-64dd-4ef8-be97-f099264e9890-host-slash\") pod \"iptables-alerter-xxx5d\" (UID: \"d74ea046-64dd-4ef8-be97-f099264e9890\") " pod="openshift-network-operator/iptables-alerter-xxx5d" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533334 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55cbdaea-d9a9-4eca-b1c0-65de273699dd-dbus\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.533362 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.537530 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c50bca7-78a9-43b3-87a3-74812b70f11b-ovnkube-config\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.537593 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-cni-binary-copy\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.537641 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-node-log\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.537674 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-run-openvswitch\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.537681 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.537683 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d800e91f-1436-4859-83e2-9bd813856736-host\") pod \"node-ca-cxmc9\" (UID: \"d800e91f-1436-4859-83e2-9bd813856736\") " pod="openshift-image-registry/node-ca-cxmc9" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.537980 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-cni-bin\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.538500 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.538535 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-cni-binary-copy\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.538617 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d800e91f-1436-4859-83e2-9bd813856736-serviceca\") pod \"node-ca-cxmc9\" (UID: \"d800e91f-1436-4859-83e2-9bd813856736\") " pod="openshift-image-registry/node-ca-cxmc9" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.538780 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3-hosts-file\") pod \"node-resolver-mfx87\" (UID: \"4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3\") " pod="openshift-dns/node-resolver-mfx87" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.538943 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4ba3c30d-3d5a-463b-918b-555fe7292755-konnectivity-ca\") pod \"konnectivity-agent-bl9f9\" (UID: \"4ba3c30d-3d5a-463b-918b-555fe7292755\") " pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:30.565738 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.539050 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-node-log\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.539096 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-systemd-units\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.539169 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-run-systemd\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.539192 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/55cbdaea-d9a9-4eca-b1c0-65de273699dd-dbus\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.539190 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.539271 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3-tmp-dir\") pod \"node-resolver-mfx87\" (UID: \"4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3\") " pod="openshift-dns/node-resolver-mfx87" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.539286 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.539381 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret podName:55cbdaea-d9a9-4eca-b1c0-65de273699dd nodeName:}" failed. No retries permitted until 2026-07-18 00:40:31.039356831 +0000 UTC m=+2.061491596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret") pod "global-pull-secret-syncer-vhhcd" (UID: "55cbdaea-d9a9-4eca-b1c0-65de273699dd") : object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.539935 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c50bca7-78a9-43b3-87a3-74812b70f11b-ovnkube-config\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.540175 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d74ea046-64dd-4ef8-be97-f099264e9890-iptables-alerter-script\") pod \"iptables-alerter-xxx5d\" (UID: \"d74ea046-64dd-4ef8-be97-f099264e9890\") " pod="openshift-network-operator/iptables-alerter-xxx5d" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.540219 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d74ea046-64dd-4ef8-be97-f099264e9890-host-slash\") pod \"iptables-alerter-xxx5d\" (UID: \"d74ea046-64dd-4ef8-be97-f099264e9890\") " pod="openshift-network-operator/iptables-alerter-xxx5d" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.538713 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d74ea046-64dd-4ef8-be97-f099264e9890-iptables-alerter-script\") pod \"iptables-alerter-xxx5d\" (UID: \"d74ea046-64dd-4ef8-be97-f099264e9890\") " pod="openshift-network-operator/iptables-alerter-xxx5d" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541076 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-kubelet\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541119 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-etc-openvswitch\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.566890 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541192 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-kubelet\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541264 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-etc-openvswitch\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541312 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9xv\" (UniqueName: \"kubernetes.io/projected/1c50bca7-78a9-43b3-87a3-74812b70f11b-kube-api-access-9x9xv\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541392 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-run-ovn\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541405 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c50bca7-78a9-43b3-87a3-74812b70f11b-ovn-node-metrics-cert\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541424 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541461 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c50bca7-78a9-43b3-87a3-74812b70f11b-env-overrides\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541759 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-run-ovn\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541852 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c50bca7-78a9-43b3-87a3-74812b70f11b-env-overrides\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541953 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.541960 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlgj6\" (UniqueName: \"kubernetes.io/projected/d800e91f-1436-4859-83e2-9bd813856736-kube-api-access-rlgj6\") pod \"node-ca-cxmc9\" (UID: \"d800e91f-1436-4859-83e2-9bd813856736\") " pod="openshift-image-registry/node-ca-cxmc9" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542046 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-os-release\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542078 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-log-socket\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542132 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-slash\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542164 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-cnibin\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542193 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55cbdaea-d9a9-4eca-b1c0-65de273699dd-kubelet-config\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542227 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-run-netns\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542260 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c50bca7-78a9-43b3-87a3-74812b70f11b-ovnkube-script-lib\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542268 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-log-socket\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542296 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-run-ovn-kubernetes\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542324 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542356 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-slash\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542373 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-cnibin\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542396 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4ba3c30d-3d5a-463b-918b-555fe7292755-agent-certs\") pod \"konnectivity-agent-bl9f9\" (UID: \"4ba3c30d-3d5a-463b-918b-555fe7292755\") " pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542454 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-var-lib-openvswitch\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542479 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542503 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-cni-netd\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542540 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-system-cni-dir\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.568933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542586 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srzxs\" (UniqueName: \"kubernetes.io/projected/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-kube-api-access-srzxs\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542679 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l72tx\" (UniqueName: \"kubernetes.io/projected/d74ea046-64dd-4ef8-be97-f099264e9890-kube-api-access-l72tx\") pod \"iptables-alerter-xxx5d\" (UID: \"d74ea046-64dd-4ef8-be97-f099264e9890\") " pod="openshift-network-operator/iptables-alerter-xxx5d" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542717 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-var-lib-openvswitch\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542781 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/55cbdaea-d9a9-4eca-b1c0-65de273699dd-kubelet-config\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542815 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-cni-netd\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542883 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-os-release\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.542954 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-run-netns\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.543086 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-system-cni-dir\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.543400 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c50bca7-78a9-43b3-87a3-74812b70f11b-host-run-ovn-kubernetes\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.543512 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c50bca7-78a9-43b3-87a3-74812b70f11b-ovnkube-script-lib\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.545532 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4ba3c30d-3d5a-463b-918b-555fe7292755-agent-certs\") pod \"konnectivity-agent-bl9f9\" (UID: \"4ba3c30d-3d5a-463b-918b-555fe7292755\") " pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.550006 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7wmg\" (UniqueName: \"kubernetes.io/projected/4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3-kube-api-access-m7wmg\") pod \"node-resolver-mfx87\" (UID: \"4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3\") " pod="openshift-dns/node-resolver-mfx87" Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.563020 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.563046 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 00:40:30.572719 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.563059 2585 projected.go:196] Error preparing data for projected volume kube-api-access-cx9j9 for pod openshift-network-diagnostics/network-check-target-hdxtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:30.573271 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.563124 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9 podName:01540c57-73db-4206-bdd9-d83fbcbb8066 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:31.06310424 +0000 UTC m=+2.085239022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cx9j9" (UniqueName: "kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9") pod "network-check-target-hdxtn" (UID: "01540c57-73db-4206-bdd9-d83fbcbb8066") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:30.573271 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.565889 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlgj6\" (UniqueName: \"kubernetes.io/projected/d800e91f-1436-4859-83e2-9bd813856736-kube-api-access-rlgj6\") pod \"node-ca-cxmc9\" (UID: \"d800e91f-1436-4859-83e2-9bd813856736\") " pod="openshift-image-registry/node-ca-cxmc9" Jul 18 00:40:30.573271 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.567076 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l72tx\" (UniqueName: \"kubernetes.io/projected/d74ea046-64dd-4ef8-be97-f099264e9890-kube-api-access-l72tx\") pod \"iptables-alerter-xxx5d\" (UID: \"d74ea046-64dd-4ef8-be97-f099264e9890\") " pod="openshift-network-operator/iptables-alerter-xxx5d" Jul 18 00:40:30.573271 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.567513 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srzxs\" (UniqueName: \"kubernetes.io/projected/513eeb46-6cbf-4b38-a4a6-e9fcc1f93767-kube-api-access-srzxs\") pod \"multus-additional-cni-plugins-p729n\" (UID: \"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767\") " pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.664679 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.574084 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x9xv\" (UniqueName: \"kubernetes.io/projected/1c50bca7-78a9-43b3-87a3-74812b70f11b-kube-api-access-9x9xv\") pod \"ovnkube-node-c6t5d\" (UID: \"1c50bca7-78a9-43b3-87a3-74812b70f11b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.664679 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.582739 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:30.664679 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.605661 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mfx87" Jul 18 00:40:30.664679 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.611355 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cxmc9" Jul 18 00:40:30.777760 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:30.709082 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae8633b8e71ee520faef372c752a6b8.slice/crio-8921240b7a4bacc99826884a6365c0772503d777bdf58ee7d6541ce50784543d WatchSource:0}: Error finding container 8921240b7a4bacc99826884a6365c0772503d777bdf58ee7d6541ce50784543d: Status 404 returned error can't find the container with id 8921240b7a4bacc99826884a6365c0772503d777bdf58ee7d6541ce50784543d Jul 18 00:40:30.777760 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:30.709626 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee0ba5c47453590857789475d8c354a6.slice/crio-4dc7759707d44e603cf54b20fe0e44edb4dd614c77abb926106d920a08431c58 WatchSource:0}: Error finding container 4dc7759707d44e603cf54b20fe0e44edb4dd614c77abb926106d920a08431c58: Status 404 returned error can't find the container with id 4dc7759707d44e603cf54b20fe0e44edb4dd614c77abb926106d920a08431c58 Jul 18 00:40:30.777760 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.710858 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" Jul 18 00:40:30.777760 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.717702 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 00:40:30.777760 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:30.721809 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634714a5_7e1e_4a53_97c7_244117cae3cd.slice/crio-78832fc9ec72ab18e7cf6e79cbcf4c2c0fe0c7b326cf6497017e44ce21cecf66 WatchSource:0}: Error finding container 78832fc9ec72ab18e7cf6e79cbcf4c2c0fe0c7b326cf6497017e44ce21cecf66: Status 404 returned error can't find the container with id 78832fc9ec72ab18e7cf6e79cbcf4c2c0fe0c7b326cf6497017e44ce21cecf66 Jul 18 00:40:30.777760 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.734323 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fft7p" Jul 18 00:40:30.777760 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.743445 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 00:40:30.777760 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:30.750421 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode18494be_47e6_488a_ae19_adbb9fe95263.slice/crio-849acb3b5e44f5c8c9b8f4ca7316dcbc4c12a0fa1ca638884c60e568e499b1e6 WatchSource:0}: Error finding container 849acb3b5e44f5c8c9b8f4ca7316dcbc4c12a0fa1ca638884c60e568e499b1e6: Status 404 returned error can't find the container with id 849acb3b5e44f5c8c9b8f4ca7316dcbc4c12a0fa1ca638884c60e568e499b1e6 Jul 18 00:40:30.777760 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.758690 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-njl82" Jul 18 00:40:30.777760 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.766856 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p729n" Jul 18 00:40:30.777760 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:30.767342 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb82a0d6_cda4_453c_8659_f3915518575a.slice/crio-4c127684fbb5b1e8aade8ddd5114553556e2b3220513addeab5ca8efb92d5583 WatchSource:0}: Error finding container 4c127684fbb5b1e8aade8ddd5114553556e2b3220513addeab5ca8efb92d5583: Status 404 returned error can't find the container with id 4c127684fbb5b1e8aade8ddd5114553556e2b3220513addeab5ca8efb92d5583 Jul 18 00:40:30.894944 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.797604 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:30.894944 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.808976 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xxx5d" Jul 18 00:40:30.894944 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:30.816420 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ba3c30d_3d5a_463b_918b_555fe7292755.slice/crio-745afae6fda16db46412faf4ed101aa706fea26fdc96d8bbc03f46193f77ac22 WatchSource:0}: Error finding container 745afae6fda16db46412faf4ed101aa706fea26fdc96d8bbc03f46193f77ac22: Status 404 returned error can't find the container with id 745afae6fda16db46412faf4ed101aa706fea26fdc96d8bbc03f46193f77ac22 Jul 18 00:40:30.894944 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:30.820877 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd74ea046_64dd_4ef8_be97_f099264e9890.slice/crio-afdd501ea5b14dcd0a61ef87393d9c248d87bb1f64b7f4fef3a0b2e1cd1928d8 WatchSource:0}: Error finding container afdd501ea5b14dcd0a61ef87393d9c248d87bb1f64b7f4fef3a0b2e1cd1928d8: Status 404 returned error can't find the container with id afdd501ea5b14dcd0a61ef87393d9c248d87bb1f64b7f4fef3a0b2e1cd1928d8 Jul 18 00:40:31.055439 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:30.930439 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c50bca7_78a9_43b3_87a3_74812b70f11b.slice/crio-295de124b6fba10efad658c5bb6a2fce717c6058c76d6bf1b5ddad019c380c04 WatchSource:0}: Error finding container 295de124b6fba10efad658c5bb6a2fce717c6058c76d6bf1b5ddad019c380c04: Status 404 returned error can't find the container with id 295de124b6fba10efad658c5bb6a2fce717c6058c76d6bf1b5ddad019c380c04 Jul 18 00:40:31.055439 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:30.945536 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:31.055439 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.945676 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:31.055439 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:30.945737 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs podName:a8f576bc-9ad2-47cd-a879-43ac2e334a95 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:31.945720181 +0000 UTC m=+2.967854945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs") pod "network-metrics-daemon-2zfkq" (UID: "a8f576bc-9ad2-47cd-a879-43ac2e334a95") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:31.055439 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:31.033503 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9721fd_fe3c_4f2d_8d09_24a59fc24ac3.slice/crio-5be101efcfb69965757bd25f582b82aa6c2cf53298eccc3b07723ccd2864d7a2 WatchSource:0}: Error finding container 5be101efcfb69965757bd25f582b82aa6c2cf53298eccc3b07723ccd2864d7a2: Status 404 returned error can't find the container with id 5be101efcfb69965757bd25f582b82aa6c2cf53298eccc3b07723ccd2864d7a2 Jul 18 00:40:31.055439 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.046162 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:31.055439 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:31.046309 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:31.055439 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:31.046370 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret podName:55cbdaea-d9a9-4eca-b1c0-65de273699dd nodeName:}" failed. No retries permitted until 2026-07-18 00:40:32.046349742 +0000 UTC m=+3.068484505 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret") pod "global-pull-secret-syncer-vhhcd" (UID: "55cbdaea-d9a9-4eca-b1c0-65de273699dd") : object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:31.172891 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:40:31.085485 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd800e91f_1436_4859_83e2_9bd813856736.slice/crio-0cdb584b1d1bd7ff53157d5e892511f21b092878275271e6ebe270394a54c766 WatchSource:0}: Error finding container 0cdb584b1d1bd7ff53157d5e892511f21b092878275271e6ebe270394a54c766: Status 404 returned error can't find the container with id 0cdb584b1d1bd7ff53157d5e892511f21b092878275271e6ebe270394a54c766 Jul 18 00:40:31.172891 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.147327 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9j9\" (UniqueName: \"kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9\") pod \"network-check-target-hdxtn\" (UID: \"01540c57-73db-4206-bdd9-d83fbcbb8066\") " pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:31.172891 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:31.147510 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 00:40:31.172891 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:31.147534 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 00:40:31.172891 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:31.147544 2585 projected.go:196] Error preparing data for projected volume kube-api-access-cx9j9 for pod openshift-network-diagnostics/network-check-target-hdxtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:31.172891 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:31.147608 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9 podName:01540c57-73db-4206-bdd9-d83fbcbb8066 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:32.147591708 +0000 UTC m=+3.169726475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cx9j9" (UniqueName: "kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9") pod "network-check-target-hdxtn" (UID: "01540c57-73db-4206-bdd9-d83fbcbb8066") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:31.291752 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.284359 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 00:40:31.417840 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.386141 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 00:35:31 +0000 UTC" deadline="2028-05-03 17:54:10.916080004 +0000 UTC" Jul 18 00:40:31.417840 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.386170 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15737h13m39.529914283s" Jul 18 00:40:31.525748 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.525612 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xxx5d" event={"ID":"d74ea046-64dd-4ef8-be97-f099264e9890","Type":"ContainerStarted","Data":"afdd501ea5b14dcd0a61ef87393d9c248d87bb1f64b7f4fef3a0b2e1cd1928d8"} Jul 18 00:40:31.530469 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.529128 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" event={"ID":"fae8633b8e71ee520faef372c752a6b8","Type":"ContainerStarted","Data":"8921240b7a4bacc99826884a6365c0772503d777bdf58ee7d6541ce50784543d"} Jul 18 00:40:31.531593 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.531493 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" event={"ID":"ee0ba5c47453590857789475d8c354a6","Type":"ContainerStarted","Data":"4dc7759707d44e603cf54b20fe0e44edb4dd614c77abb926106d920a08431c58"} Jul 18 00:40:31.533346 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.533215 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cxmc9" event={"ID":"d800e91f-1436-4859-83e2-9bd813856736","Type":"ContainerStarted","Data":"0cdb584b1d1bd7ff53157d5e892511f21b092878275271e6ebe270394a54c766"} Jul 18 00:40:31.537577 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.537474 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mfx87" event={"ID":"4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3","Type":"ContainerStarted","Data":"5be101efcfb69965757bd25f582b82aa6c2cf53298eccc3b07723ccd2864d7a2"} Jul 18 00:40:31.540887 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.540755 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" event={"ID":"1c50bca7-78a9-43b3-87a3-74812b70f11b","Type":"ContainerStarted","Data":"295de124b6fba10efad658c5bb6a2fce717c6058c76d6bf1b5ddad019c380c04"} Jul 18 00:40:31.545484 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.545399 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bl9f9" event={"ID":"4ba3c30d-3d5a-463b-918b-555fe7292755","Type":"ContainerStarted","Data":"745afae6fda16db46412faf4ed101aa706fea26fdc96d8bbc03f46193f77ac22"} Jul 18 00:40:31.554110 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.554023 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p729n" event={"ID":"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767","Type":"ContainerStarted","Data":"521d4921d8e9b4632d4375c85dc3d61524989131c53068a643915b55580bfcf8"} Jul 18 00:40:31.565528 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.565428 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-njl82" event={"ID":"db82a0d6-cda4-453c-8659-f3915518575a","Type":"ContainerStarted","Data":"4c127684fbb5b1e8aade8ddd5114553556e2b3220513addeab5ca8efb92d5583"} Jul 18 00:40:31.570942 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.570832 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fft7p" event={"ID":"e18494be-47e6-488a-ae19-adbb9fe95263","Type":"ContainerStarted","Data":"849acb3b5e44f5c8c9b8f4ca7316dcbc4c12a0fa1ca638884c60e568e499b1e6"} Jul 18 00:40:31.572998 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.572886 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" event={"ID":"634714a5-7e1e-4a53-97c7-244117cae3cd","Type":"ContainerStarted","Data":"78832fc9ec72ab18e7cf6e79cbcf4c2c0fe0c7b326cf6497017e44ce21cecf66"} Jul 18 00:40:31.955399 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:31.955345 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:31.955614 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:31.955526 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:31.955614 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:31.955598 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs podName:a8f576bc-9ad2-47cd-a879-43ac2e334a95 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:33.955575374 +0000 UTC m=+4.977710143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs") pod "network-metrics-daemon-2zfkq" (UID: "a8f576bc-9ad2-47cd-a879-43ac2e334a95") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:32.056856 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:32.056150 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:32.056856 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:32.056314 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:32.056856 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:32.056389 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret podName:55cbdaea-d9a9-4eca-b1c0-65de273699dd nodeName:}" failed. No retries permitted until 2026-07-18 00:40:34.056367632 +0000 UTC m=+5.078502448 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret") pod "global-pull-secret-syncer-vhhcd" (UID: "55cbdaea-d9a9-4eca-b1c0-65de273699dd") : object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:32.157709 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:32.157663 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9j9\" (UniqueName: \"kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9\") pod \"network-check-target-hdxtn\" (UID: \"01540c57-73db-4206-bdd9-d83fbcbb8066\") " pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:32.157974 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:32.157933 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 00:40:32.157974 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:32.157963 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 00:40:32.157974 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:32.157977 2585 projected.go:196] Error preparing data for projected volume kube-api-access-cx9j9 for pod openshift-network-diagnostics/network-check-target-hdxtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:32.158170 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:32.158042 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9 podName:01540c57-73db-4206-bdd9-d83fbcbb8066 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:34.158022503 +0000 UTC m=+5.180157270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cx9j9" (UniqueName: "kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9") pod "network-check-target-hdxtn" (UID: "01540c57-73db-4206-bdd9-d83fbcbb8066") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:32.387221 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:32.387031 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 00:35:31 +0000 UTC" deadline="2028-04-12 15:44:48.775482852 +0000 UTC" Jul 18 00:40:32.387221 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:32.387105 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15231h4m16.388382911s" Jul 18 00:40:32.426589 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:32.426170 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 00:40:32.451279 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:32.451191 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:32.451279 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:32.451219 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:32.451279 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:32.451255 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:32.451587 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:32.451413 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:32.451881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:32.451785 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:32.452011 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:32.451867 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:33.974690 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:33.974636 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:33.975374 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:33.974800 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:33.975374 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:33.974891 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs podName:a8f576bc-9ad2-47cd-a879-43ac2e334a95 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:37.974852062 +0000 UTC m=+8.996986830 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs") pod "network-metrics-daemon-2zfkq" (UID: "a8f576bc-9ad2-47cd-a879-43ac2e334a95") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:34.077510 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:34.075723 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:34.077510 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:34.075940 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:34.077510 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:34.076008 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret podName:55cbdaea-d9a9-4eca-b1c0-65de273699dd nodeName:}" failed. No retries permitted until 2026-07-18 00:40:38.075988726 +0000 UTC m=+9.098123496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret") pod "global-pull-secret-syncer-vhhcd" (UID: "55cbdaea-d9a9-4eca-b1c0-65de273699dd") : object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:34.177069 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:34.176965 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9j9\" (UniqueName: \"kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9\") pod \"network-check-target-hdxtn\" (UID: \"01540c57-73db-4206-bdd9-d83fbcbb8066\") " pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:34.177268 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:34.177198 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 00:40:34.177268 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:34.177227 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 00:40:34.177268 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:34.177245 2585 projected.go:196] Error preparing data for projected volume kube-api-access-cx9j9 for pod openshift-network-diagnostics/network-check-target-hdxtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:34.177442 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:34.177315 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9 podName:01540c57-73db-4206-bdd9-d83fbcbb8066 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:38.177292994 +0000 UTC m=+9.199427758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cx9j9" (UniqueName: "kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9") pod "network-check-target-hdxtn" (UID: "01540c57-73db-4206-bdd9-d83fbcbb8066") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:34.451684 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:34.451646 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:34.451855 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:34.451794 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:34.452940 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:34.452280 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:34.452940 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:34.452526 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:34.452940 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:34.452651 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:34.452940 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:34.452818 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:36.451889 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:36.450978 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:36.451889 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:36.451146 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:36.451889 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:36.451599 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:36.451889 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:36.451695 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:36.451889 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:36.451760 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:36.451889 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:36.451835 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:38.014226 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:38.014145 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:38.014746 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:38.014287 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:38.014746 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:38.014373 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs podName:a8f576bc-9ad2-47cd-a879-43ac2e334a95 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:46.014349084 +0000 UTC m=+17.036483864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs") pod "network-metrics-daemon-2zfkq" (UID: "a8f576bc-9ad2-47cd-a879-43ac2e334a95") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:38.115407 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:38.115364 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:38.115573 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:38.115531 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:38.115627 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:38.115599 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret podName:55cbdaea-d9a9-4eca-b1c0-65de273699dd nodeName:}" failed. No retries permitted until 2026-07-18 00:40:46.115581067 +0000 UTC m=+17.137715831 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret") pod "global-pull-secret-syncer-vhhcd" (UID: "55cbdaea-d9a9-4eca-b1c0-65de273699dd") : object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:38.216514 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:38.216471 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9j9\" (UniqueName: \"kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9\") pod \"network-check-target-hdxtn\" (UID: \"01540c57-73db-4206-bdd9-d83fbcbb8066\") " pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:38.216723 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:38.216632 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 00:40:38.216723 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:38.216659 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 00:40:38.216723 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:38.216672 2585 projected.go:196] Error preparing data for projected volume kube-api-access-cx9j9 for pod openshift-network-diagnostics/network-check-target-hdxtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:38.216939 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:38.216740 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9 podName:01540c57-73db-4206-bdd9-d83fbcbb8066 nodeName:}" failed. No retries permitted until 2026-07-18 00:40:46.216720128 +0000 UTC m=+17.238854893 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cx9j9" (UniqueName: "kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9") pod "network-check-target-hdxtn" (UID: "01540c57-73db-4206-bdd9-d83fbcbb8066") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:38.451785 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:38.451285 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:38.451785 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:38.451404 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:38.451785 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:38.451413 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:38.451785 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:38.451542 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:38.451785 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:38.451595 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:38.451785 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:38.451666 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:40.451400 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:40.451298 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:40.451400 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:40.451328 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:40.452004 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:40.451432 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:40.452004 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:40.451824 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:40.452004 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:40.451961 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:40.452159 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:40.452053 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:42.451260 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:42.451220 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:42.451260 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:42.451243 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:42.451712 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:42.451336 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:42.451712 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:42.451356 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:42.451712 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:42.451446 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:42.451712 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:42.451538 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:44.450866 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:44.450762 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:44.450866 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:44.450812 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:44.450866 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:44.450812 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:44.451394 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:44.450954 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:44.451394 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:44.451087 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:44.451394 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:44.451169 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:46.082241 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:46.082196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:46.082722 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:46.082319 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:46.082722 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:46.082382 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs podName:a8f576bc-9ad2-47cd-a879-43ac2e334a95 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:02.082365819 +0000 UTC m=+33.104500581 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs") pod "network-metrics-daemon-2zfkq" (UID: "a8f576bc-9ad2-47cd-a879-43ac2e334a95") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:40:46.182944 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:46.182899 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:46.183123 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:46.183070 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:46.183197 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:46.183150 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret podName:55cbdaea-d9a9-4eca-b1c0-65de273699dd nodeName:}" failed. No retries permitted until 2026-07-18 00:41:02.183129554 +0000 UTC m=+33.205264318 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret") pod "global-pull-secret-syncer-vhhcd" (UID: "55cbdaea-d9a9-4eca-b1c0-65de273699dd") : object "kube-system"/"original-pull-secret" not registered Jul 18 00:40:46.283452 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:46.283413 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9j9\" (UniqueName: \"kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9\") pod \"network-check-target-hdxtn\" (UID: \"01540c57-73db-4206-bdd9-d83fbcbb8066\") " pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:46.283653 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:46.283584 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 00:40:46.283653 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:46.283606 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 00:40:46.283653 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:46.283616 2585 projected.go:196] Error preparing data for projected volume kube-api-access-cx9j9 for pod openshift-network-diagnostics/network-check-target-hdxtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:46.283797 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:46.283676 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9 podName:01540c57-73db-4206-bdd9-d83fbcbb8066 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:02.283655932 +0000 UTC m=+33.305790742 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cx9j9" (UniqueName: "kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9") pod "network-check-target-hdxtn" (UID: "01540c57-73db-4206-bdd9-d83fbcbb8066") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:40:46.451419 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:46.451332 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:46.451419 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:46.451384 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:46.451642 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:46.451330 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:46.451642 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:46.451471 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:46.451642 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:46.451589 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:46.451792 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:46.451675 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:48.450990 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:48.450959 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:48.451412 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:48.450960 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:48.451412 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:48.451091 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:48.451412 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:48.450960 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:48.451412 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:48.451154 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:48.451412 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:48.451262 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:50.450614 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:50.450570 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:50.451105 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:50.450569 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:50.451105 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:50.450708 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:50.451105 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:50.450570 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:50.451105 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:50.450790 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:50.451105 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:50.450868 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:52.450780 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.450750 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:52.451209 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.450791 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:52.451209 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.450862 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:52.451209 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:52.450879 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:52.451209 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:52.450942 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:52.451209 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:52.450987 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:52.653213 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.652923 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-njl82" event={"ID":"db82a0d6-cda4-453c-8659-f3915518575a","Type":"ContainerStarted","Data":"d536e2e6ac81f92c6fcd65d427dc6c1d9fa80a3297eaec489be0e807894051a6"} Jul 18 00:40:52.654756 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.654722 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fft7p" event={"ID":"e18494be-47e6-488a-ae19-adbb9fe95263","Type":"ContainerStarted","Data":"4550e10c964bee0a1e121b468e024b7cebb444651fe3d66aa21e3b4aa5abf5de"} Jul 18 00:40:52.656493 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.656437 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" event={"ID":"ee0ba5c47453590857789475d8c354a6","Type":"ContainerStarted","Data":"8edb491cb5a5c96817b9efaca9d4e253d88db4e95676fa20ef298a6cc423be3a"} Jul 18 00:40:52.659443 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.659420 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 00:40:52.659769 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.659745 2585 generic.go:356] "Generic (PLEG): container finished" podID="1c50bca7-78a9-43b3-87a3-74812b70f11b" containerID="95fe4af00d0b2f2674b5ccdf4451126f962d81851f4fd7d20818ea1639329b94" exitCode=1 Jul 18 00:40:52.659874 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.659781 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" event={"ID":"1c50bca7-78a9-43b3-87a3-74812b70f11b","Type":"ContainerStarted","Data":"8042aaba4d56f59660d7897a71aa249239b691f5446f32ed4543fa35cb65423b"} Jul 18 00:40:52.659874 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.659849 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" event={"ID":"1c50bca7-78a9-43b3-87a3-74812b70f11b","Type":"ContainerStarted","Data":"ad8853eda2c66bc6d6bfe11ef2954fa1778cfe968a7920ae9431f86460cf13ce"} Jul 18 00:40:52.660058 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.659882 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" event={"ID":"1c50bca7-78a9-43b3-87a3-74812b70f11b","Type":"ContainerStarted","Data":"0f5d611dcb25fb39656e6623b2778cb5c23bdb0a3a69c1cebd66730d6b818740"} Jul 18 00:40:52.660058 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.659911 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" event={"ID":"1c50bca7-78a9-43b3-87a3-74812b70f11b","Type":"ContainerDied","Data":"95fe4af00d0b2f2674b5ccdf4451126f962d81851f4fd7d20818ea1639329b94"} Jul 18 00:40:52.660058 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.659940 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" event={"ID":"1c50bca7-78a9-43b3-87a3-74812b70f11b","Type":"ContainerStarted","Data":"75c9a3f43d65230b7d0511e4c6372ae4eb93d65bcf1a4d076475a51fbf8373ce"} Jul 18 00:40:52.669282 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.669237 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-njl82" podStartSLOduration=1.519350105 podStartE2EDuration="22.669223172s" totalImagesPullingTime="21.149873067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:52.669159393 +0000 UTC m=+23.691294179" watchObservedRunningTime="2026-07-18 00:40:52.669223172 +0000 UTC m=+23.691357955" Jul 18 00:40:52.683080 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.683005 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-148.ec2.internal" podStartSLOduration=21.682986664 podStartE2EDuration="21.682986664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:52.682646384 +0000 UTC m=+23.704781169" watchObservedRunningTime="2026-07-18 00:40:52.682986664 +0000 UTC m=+23.705121450" Jul 18 00:40:52.702306 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:52.702263 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fft7p" podStartSLOduration=1.4792989539999999 podStartE2EDuration="22.702247272s" totalImagesPullingTime="21.222948318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:52.701784907 +0000 UTC m=+23.723919691" watchObservedRunningTime="2026-07-18 00:40:52.702247272 +0000 UTC m=+23.724382057" Jul 18 00:40:53.663904 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.663653 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cxmc9" event={"ID":"d800e91f-1436-4859-83e2-9bd813856736","Type":"ContainerStarted","Data":"ae02076cc4f811a5d0324806623303715fa95f17c62a256fd1086f52ca94d5f0"} Jul 18 00:40:53.665585 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.665553 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mfx87" event={"ID":"4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3","Type":"ContainerStarted","Data":"a9dc404f49de133131c163350a32bd003bed42fc46d2fba70a73930f9e9c2f13"} Jul 18 00:40:53.668851 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.668819 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 00:40:53.669253 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.669227 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" event={"ID":"1c50bca7-78a9-43b3-87a3-74812b70f11b","Type":"ContainerStarted","Data":"bd972ad5e55e5305a2e3659e5dbf7e0c14de12663228ed8b5b40d93bfcc3effe"} Jul 18 00:40:53.670823 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.670792 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bl9f9" event={"ID":"4ba3c30d-3d5a-463b-918b-555fe7292755","Type":"ContainerStarted","Data":"35485264202ab38c904307b78e03cf7fb2f935d86c22811d2ece90296e0d49da"} Jul 18 00:40:53.672389 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.672367 2585 generic.go:356] "Generic (PLEG): container finished" podID="513eeb46-6cbf-4b38-a4a6-e9fcc1f93767" containerID="53f598840c393350b66622083a59f825c73da20f820a0a40a4b7fe31f99a7e48" exitCode=0 Jul 18 00:40:53.672495 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.672443 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p729n" event={"ID":"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767","Type":"ContainerDied","Data":"53f598840c393350b66622083a59f825c73da20f820a0a40a4b7fe31f99a7e48"} Jul 18 00:40:53.674041 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.674013 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" event={"ID":"634714a5-7e1e-4a53-97c7-244117cae3cd","Type":"ContainerStarted","Data":"e9c39c52ff9bea5010e994a96d6bfec722c803d1bd1ca1400a5f2732b572c280"} Jul 18 00:40:53.675644 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.675627 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xxx5d" event={"ID":"d74ea046-64dd-4ef8-be97-f099264e9890","Type":"ContainerStarted","Data":"a59b844fa9563fbe3518ac358a29797625f9941dea42e22f91fb0e65b7cb3876"} Jul 18 00:40:53.677168 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.677148 2585 generic.go:356] "Generic (PLEG): container finished" podID="fae8633b8e71ee520faef372c752a6b8" containerID="6f56869cc6c0e582ae2f3cf672e58e3a9bd5f58cb756e5df4187a118acbf7bbf" exitCode=0 Jul 18 00:40:53.677274 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.677222 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" event={"ID":"fae8633b8e71ee520faef372c752a6b8","Type":"ContainerDied","Data":"6f56869cc6c0e582ae2f3cf672e58e3a9bd5f58cb756e5df4187a118acbf7bbf"} Jul 18 00:40:53.678787 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.678750 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cxmc9" podStartSLOduration=2.855979239 podStartE2EDuration="23.678738858s" totalImagesPullingTime="20.822759619s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:53.678454375 +0000 UTC m=+24.700589158" watchObservedRunningTime="2026-07-18 00:40:53.678738858 +0000 UTC m=+24.700873643" Jul 18 00:40:53.692001 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.691946 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bl9f9" podStartSLOduration=2.594730981 podStartE2EDuration="23.69193348s" totalImagesPullingTime="21.097202499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:53.691286918 +0000 UTC m=+24.713421714" watchObservedRunningTime="2026-07-18 00:40:53.69193348 +0000 UTC m=+24.714068263" Jul 18 00:40:53.705868 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.705802 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xxx5d" podStartSLOduration=2.6220998460000002 podStartE2EDuration="23.70578889s" totalImagesPullingTime="21.083689044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:53.705327156 +0000 UTC m=+24.727461942" watchObservedRunningTime="2026-07-18 00:40:53.70578889 +0000 UTC m=+24.727923676" Jul 18 00:40:53.738766 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:53.738722 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mfx87" podStartSLOduration=2.852603491 podStartE2EDuration="23.738708453s" totalImagesPullingTime="20.886104962s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:53.738398263 +0000 UTC m=+24.760533047" watchObservedRunningTime="2026-07-18 00:40:53.738708453 +0000 UTC m=+24.760843233" Jul 18 00:40:54.174287 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.174260 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 18 00:40:54.415479 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.415334 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T00:40:54.174276244Z","UUID":"9d48dac4-d29f-45ea-b115-560dd993ea56","Handler":null,"Name":"","Endpoint":""} Jul 18 00:40:54.418488 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.418463 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 18 00:40:54.418634 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.418509 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 18 00:40:54.450921 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.450886 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:54.450921 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.450912 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:54.451153 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.450917 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:54.451153 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:54.451015 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:54.451153 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:54.451100 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:54.451312 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:54.451180 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:54.682011 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.681916 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" event={"ID":"634714a5-7e1e-4a53-97c7-244117cae3cd","Type":"ContainerStarted","Data":"037771e259638017ce9af8bf1d914500405305326c8c9a01e3518c08eeeffddb"} Jul 18 00:40:54.684148 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.684117 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" event={"ID":"fae8633b8e71ee520faef372c752a6b8","Type":"ContainerStarted","Data":"486d36e2a7cbb6f14818daeed01de31dc2bb8ad90c9eec2e99fa158daa1520e1"} Jul 18 00:40:54.847239 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.847205 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:54.848426 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.848405 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:54.865479 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:54.865424 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-148.ec2.internal" podStartSLOduration=23.865403731 podStartE2EDuration="23.865403731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:54.720921855 +0000 UTC m=+25.743056640" watchObservedRunningTime="2026-07-18 00:40:54.865403731 +0000 UTC m=+25.887538521" Jul 18 00:40:55.690747 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:55.690719 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 00:40:55.691263 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:55.691173 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" event={"ID":"1c50bca7-78a9-43b3-87a3-74812b70f11b","Type":"ContainerStarted","Data":"67f84c0b0eab993651eb937fb640ec20d54ad76b07fb83fa27bed02969f8e26f"} Jul 18 00:40:55.693434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:55.693399 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" event={"ID":"634714a5-7e1e-4a53-97c7-244117cae3cd","Type":"ContainerStarted","Data":"03a8260e3426284c4d47a0dbd88bd4fbc1cfb07ac5db9fe7ec457562aacb933d"} Jul 18 00:40:55.693982 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:55.693962 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:55.694470 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:55.694448 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bl9f9" Jul 18 00:40:55.711472 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:55.711426 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rjp78" podStartSLOduration=1.95438358 podStartE2EDuration="25.711409372s" totalImagesPullingTime="23.757025792s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:55.711116033 +0000 UTC m=+26.733250832" watchObservedRunningTime="2026-07-18 00:40:55.711409372 +0000 UTC m=+26.733544155" Jul 18 00:40:56.451458 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:56.451421 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:56.451765 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:56.451422 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:56.451765 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:56.451564 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:56.451765 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:56.451421 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:56.451765 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:56.451633 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:56.451765 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:56.451699 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:58.451398 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:58.451151 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:58.452139 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:58.451151 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:58.452139 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:58.451439 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:58.452139 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:58.451151 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:58.452139 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:58.451492 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:40:58.452139 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:58.451571 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:58.703724 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:58.703650 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 00:40:58.704073 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:58.704045 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" event={"ID":"1c50bca7-78a9-43b3-87a3-74812b70f11b","Type":"ContainerStarted","Data":"5e9bf5d271e9e4f2d5449d84d32586cd7a7372038796c631fa582da5f38a58df"} Jul 18 00:40:58.704315 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:58.704293 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:58.704585 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:58.704561 2585 scope.go:117] "RemoveContainer" containerID="95fe4af00d0b2f2674b5ccdf4451126f962d81851f4fd7d20818ea1639329b94" Jul 18 00:40:58.706057 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:58.706029 2585 generic.go:356] "Generic (PLEG): container finished" podID="513eeb46-6cbf-4b38-a4a6-e9fcc1f93767" containerID="924fa386f2f4a90b206fc76dca4eac5d6275019db59faa506a67d2f8b2b9627b" exitCode=0 Jul 18 00:40:58.706151 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:58.706087 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p729n" event={"ID":"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767","Type":"ContainerDied","Data":"924fa386f2f4a90b206fc76dca4eac5d6275019db59faa506a67d2f8b2b9627b"} Jul 18 00:40:58.720965 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:58.720938 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:59.713175 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.712994 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 00:40:59.713614 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.713588 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" event={"ID":"1c50bca7-78a9-43b3-87a3-74812b70f11b","Type":"ContainerStarted","Data":"c83ce7c79b03ec3b39dfcc4aa0100ced781d119381b8b2a3ba3d42cf3a9477a0"} Jul 18 00:40:59.713977 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.713779 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:59.713977 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.713861 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:59.715799 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.715776 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p729n" event={"ID":"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767","Type":"ContainerStarted","Data":"df9e6127d4d859055952ea659bcebe0aa62c47423b27726a3fb2189bc14b8870"} Jul 18 00:40:59.729822 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.729798 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:40:59.742735 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.742689 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" podStartSLOduration=8.691997692 podStartE2EDuration="29.742674272s" totalImagesPullingTime="21.05067658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:40:59.741168901 +0000 UTC m=+30.763303681" watchObservedRunningTime="2026-07-18 00:40:59.742674272 +0000 UTC m=+30.764809056" Jul 18 00:40:59.787305 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.787271 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2zfkq"] Jul 18 00:40:59.787494 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.787406 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:40:59.787560 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:59.787536 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:40:59.788062 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.788041 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hdxtn"] Jul 18 00:40:59.788184 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.788133 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:40:59.788380 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:59.788293 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:40:59.788754 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.788735 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vhhcd"] Jul 18 00:40:59.788853 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:40:59.788845 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:40:59.788952 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:40:59.788933 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:41:00.720674 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:00.720637 2585 generic.go:356] "Generic (PLEG): container finished" podID="513eeb46-6cbf-4b38-a4a6-e9fcc1f93767" containerID="df9e6127d4d859055952ea659bcebe0aa62c47423b27726a3fb2189bc14b8870" exitCode=0 Jul 18 00:41:00.721087 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:00.720727 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p729n" event={"ID":"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767","Type":"ContainerDied","Data":"df9e6127d4d859055952ea659bcebe0aa62c47423b27726a3fb2189bc14b8870"} Jul 18 00:41:01.451647 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:01.451619 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:41:01.451810 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:01.451620 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:41:01.451810 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:01.451736 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:41:01.451810 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:01.451620 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:41:01.452021 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:01.451865 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:41:01.452021 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:01.451918 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:41:01.726412 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:01.724929 2585 generic.go:356] "Generic (PLEG): container finished" podID="513eeb46-6cbf-4b38-a4a6-e9fcc1f93767" containerID="9ac5bdb7de807597039c6edc4238670a669fc778ba4ee3ef110084aed793c11f" exitCode=0 Jul 18 00:41:01.726412 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:01.725005 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p729n" event={"ID":"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767","Type":"ContainerDied","Data":"9ac5bdb7de807597039c6edc4238670a669fc778ba4ee3ef110084aed793c11f"} Jul 18 00:41:02.101695 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:02.101614 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:41:02.101877 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:02.101732 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:41:02.101877 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:02.101784 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs podName:a8f576bc-9ad2-47cd-a879-43ac2e334a95 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:34.101770881 +0000 UTC m=+65.123905643 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs") pod "network-metrics-daemon-2zfkq" (UID: "a8f576bc-9ad2-47cd-a879-43ac2e334a95") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 00:41:02.202400 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:02.202367 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:41:02.202573 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:02.202511 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 00:41:02.202622 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:02.202590 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret podName:55cbdaea-d9a9-4eca-b1c0-65de273699dd nodeName:}" failed. No retries permitted until 2026-07-18 00:41:34.202570934 +0000 UTC m=+65.224705697 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret") pod "global-pull-secret-syncer-vhhcd" (UID: "55cbdaea-d9a9-4eca-b1c0-65de273699dd") : object "kube-system"/"original-pull-secret" not registered Jul 18 00:41:02.302865 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:02.302816 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9j9\" (UniqueName: \"kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9\") pod \"network-check-target-hdxtn\" (UID: \"01540c57-73db-4206-bdd9-d83fbcbb8066\") " pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:41:02.303062 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:02.302998 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 00:41:02.303062 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:02.303025 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 00:41:02.303062 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:02.303038 2585 projected.go:196] Error preparing data for projected volume kube-api-access-cx9j9 for pod openshift-network-diagnostics/network-check-target-hdxtn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:41:02.303219 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:02.303106 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9 podName:01540c57-73db-4206-bdd9-d83fbcbb8066 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:34.303086528 +0000 UTC m=+65.325221292 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cx9j9" (UniqueName: "kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9") pod "network-check-target-hdxtn" (UID: "01540c57-73db-4206-bdd9-d83fbcbb8066") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 00:41:03.451532 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:03.451344 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:41:03.452025 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:03.451344 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:41:03.452025 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:03.451632 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hdxtn" podUID="01540c57-73db-4206-bdd9-d83fbcbb8066" Jul 18 00:41:03.452025 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:03.451349 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:41:03.452025 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:03.451693 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vhhcd" podUID="55cbdaea-d9a9-4eca-b1c0-65de273699dd" Jul 18 00:41:03.452025 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:03.451847 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zfkq" podUID="a8f576bc-9ad2-47cd-a879-43ac2e334a95" Jul 18 00:41:04.316252 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.316176 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-148.ec2.internal" event="NodeReady" Jul 18 00:41:04.316421 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.316329 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 18 00:41:04.369506 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.369472 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9w9q5"] Jul 18 00:41:04.388679 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.387817 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9w9q5" Jul 18 00:41:04.388679 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.387938 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-kgsmq"] Jul 18 00:41:04.392876 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.392853 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-qglkq\"" Jul 18 00:41:04.393267 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.393231 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 18 00:41:04.393456 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.393443 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 18 00:41:04.403949 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.403924 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6"] Jul 18 00:41:04.404173 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.404142 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.407075 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.406908 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 18 00:41:04.407075 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.406938 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 18 00:41:04.407075 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.406981 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 18 00:41:04.407075 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.406908 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 18 00:41:04.407392 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.407317 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-v6hjs\"" Jul 18 00:41:04.421313 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.418963 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-z987w"] Jul 18 00:41:04.421313 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.419254 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:04.424479 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.424224 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 18 00:41:04.424479 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.424261 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tz56m\"" Jul 18 00:41:04.425506 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.425477 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 18 00:41:04.425730 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.425688 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 18 00:41:04.426341 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.426325 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 18 00:41:04.441864 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.441823 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl"] Jul 18 00:41:04.441973 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.441914 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.444285 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.444264 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 18 00:41:04.444444 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.444416 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 18 00:41:04.444529 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.444482 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9tzq4\"" Jul 18 00:41:04.444594 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.444542 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 18 00:41:04.444683 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.444668 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 18 00:41:04.449554 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.449535 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 18 00:41:04.456000 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.455955 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-559bddd8f8-w24db"] Jul 18 00:41:04.456356 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.456092 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" Jul 18 00:41:04.458420 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.458400 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 18 00:41:04.458501 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.458441 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 18 00:41:04.458501 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.458400 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-8nm6d\"" Jul 18 00:41:04.458501 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.458401 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 18 00:41:04.458784 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.458768 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 18 00:41:04.470063 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.470042 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj"] Jul 18 00:41:04.470213 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.470194 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.472424 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.472405 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 18 00:41:04.472891 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.472647 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 18 00:41:04.472891 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.472736 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 18 00:41:04.472891 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.472744 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 18 00:41:04.472891 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.472749 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 18 00:41:04.472891 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.472872 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 18 00:41:04.473173 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.472876 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nfcpz\"" Jul 18 00:41:04.484130 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.484108 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5dfcf5fff6-2h49l"] Jul 18 00:41:04.484251 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.484212 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:04.487403 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.487377 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 18 00:41:04.487403 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.487399 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rgftg\"" Jul 18 00:41:04.487567 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.487413 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 18 00:41:04.487567 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.487380 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 18 00:41:04.489411 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.489368 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 18 00:41:04.496339 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.496317 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9w9q5"] Jul 18 00:41:04.496455 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.496352 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2gm4c"] Jul 18 00:41:04.496455 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.496436 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.499960 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.499935 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 18 00:41:04.500176 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.500142 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 18 00:41:04.500265 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.500194 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 18 00:41:04.500760 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.500739 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hd2wz\"" Jul 18 00:41:04.508707 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.508671 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6"] Jul 18 00:41:04.509159 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.509140 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 18 00:41:04.509222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.509168 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-kgsmq"] Jul 18 00:41:04.509222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.509177 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.509222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.509197 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl"] Jul 18 00:41:04.509222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.509221 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz"] Jul 18 00:41:04.516725 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.516709 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j7vqp\"" Jul 18 00:41:04.516851 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.516710 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 18 00:41:04.516922 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.516852 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 18 00:41:04.520432 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.520411 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54"] Jul 18 00:41:04.520770 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.520753 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" Jul 18 00:41:04.523350 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.523327 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-m9fng\"" Jul 18 00:41:04.523483 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.523464 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 18 00:41:04.523682 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.523568 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 18 00:41:04.524110 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.524091 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 18 00:41:04.524241 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.524223 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 18 00:41:04.524708 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.524685 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/602eb3d3-135b-46da-8f9a-0061de2cc17b-service-ca-bundle\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.524854 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.524734 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn525\" (UniqueName: \"kubernetes.io/projected/15ccd479-3004-4964-a298-0c510fc4bf9b-kube-api-access-rn525\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.524854 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.524761 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/602eb3d3-135b-46da-8f9a-0061de2cc17b-tmp\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.524854 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.524786 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/602eb3d3-135b-46da-8f9a-0061de2cc17b-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.524854 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.524818 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfgkv\" (UniqueName: \"kubernetes.io/projected/7fea5857-2011-457a-91ad-c830f4686d0e-kube-api-access-kfgkv\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:04.525067 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.524889 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v7s2\" (UniqueName: \"kubernetes.io/projected/e074f31c-0e2d-4e42-b988-2564e4e2d6a0-kube-api-access-6v7s2\") pod \"volume-data-source-validator-78cf4c6f6-9w9q5\" (UID: \"e074f31c-0e2d-4e42-b988-2564e4e2d6a0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9w9q5" Jul 18 00:41:04.525067 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.524910 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/15ccd479-3004-4964-a298-0c510fc4bf9b-config\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.525067 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.524952 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15ccd479-3004-4964-a298-0c510fc4bf9b-serving-cert\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.525067 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.524969 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/602eb3d3-135b-46da-8f9a-0061de2cc17b-serving-cert\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.525067 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.525031 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/602eb3d3-135b-46da-8f9a-0061de2cc17b-snapshots\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.525300 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.525137 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15ccd479-3004-4964-a298-0c510fc4bf9b-trusted-ca\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.525300 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.525177 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4pkd\" (UniqueName: \"kubernetes.io/projected/602eb3d3-135b-46da-8f9a-0061de2cc17b-kube-api-access-n4pkd\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.525300 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.525212 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:04.532482 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.532463 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-z987w"] Jul 18 00:41:04.532564 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.532500 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8n9md"] Jul 18 00:41:04.532603 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.532566 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:04.535436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.535416 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 18 00:41:04.536274 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.536256 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 18 00:41:04.536408 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.536286 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8nvk6\"" Jul 18 00:41:04.544595 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.544572 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-64kzz"] Jul 18 00:41:04.544694 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.544685 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8n9md" Jul 18 00:41:04.547173 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.547153 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-h5lw6\"" Jul 18 00:41:04.547506 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.547490 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 18 00:41:04.547575 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.547531 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 18 00:41:04.547772 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.547755 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 18 00:41:04.553354 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.553286 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:04.557313 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.557293 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-559bddd8f8-w24db"] Jul 18 00:41:04.557410 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.557322 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dfcf5fff6-2h49l"] Jul 18 00:41:04.557410 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.557337 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz"] Jul 18 00:41:04.557410 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.557353 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54"] Jul 18 00:41:04.557410 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.557369 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-64kzz"] Jul 18 00:41:04.557410 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.557381 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2gm4c"] Jul 18 00:41:04.557410 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.557396 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8n9md"] Jul 18 00:41:04.557410 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.557413 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj"] Jul 18 00:41:04.557671 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.557418 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-64kzz" Jul 18 00:41:04.559783 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.559760 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 00:41:04.559908 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.559788 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xc2sg\"" Jul 18 00:41:04.559908 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.559879 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 00:41:04.625710 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.625627 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/70547afb-1be6-4358-b0cd-5349d7dbf273-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:04.625710 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.625672 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:04.625710 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.625704 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-certificates\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.626013 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.625748 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/602eb3d3-135b-46da-8f9a-0061de2cc17b-service-ca-bundle\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.626013 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.625787 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 00:41:04.626013 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.625793 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn525\" (UniqueName: \"kubernetes.io/projected/15ccd479-3004-4964-a298-0c510fc4bf9b-kube-api-access-rn525\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.626013 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.625817 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-default-certificate\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.626013 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.625878 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:04.626013 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.625908 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls podName:7fea5857-2011-457a-91ad-c830f4686d0e nodeName:}" failed. No retries permitted until 2026-07-18 00:41:05.125889045 +0000 UTC m=+36.148023808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-67bw6" (UID: "7fea5857-2011-457a-91ad-c830f4686d0e") : secret "samples-operator-tls" not found Jul 18 00:41:04.626013 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.625935 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-image-registry-private-configuration\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.626013 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.625973 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-installation-pull-secrets\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.626013 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.625993 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cltg7\" (UniqueName: \"kubernetes.io/projected/80a66f7f-3a1d-4252-8139-846b74939b67-kube-api-access-cltg7\") pod \"ingress-canary-8n9md\" (UID: \"80a66f7f-3a1d-4252-8139-846b74939b67\") " pod="openshift-ingress-canary/ingress-canary-8n9md" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626020 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15ccd479-3004-4964-a298-0c510fc4bf9b-serving-cert\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626049 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa89d4ee-2ac6-4234-8641-a851e295524d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8j4fl\" (UID: \"aa89d4ee-2ac6-4234-8641-a851e295524d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626173 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9a89e63-02c2-495f-b2d5-72742bc08595-ca-trust-extracted\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626217 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626249 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15ccd479-3004-4964-a298-0c510fc4bf9b-trusted-ca\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626291 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f6b75f9-43a8-4d7c-9cd0-7a350a753397-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-hdjjz\" (UID: \"2f6b75f9-43a8-4d7c-9cd0-7a350a753397\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626321 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626348 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb5lr\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-kube-api-access-wb5lr\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626372 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-stats-auth\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626390 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/602eb3d3-135b-46da-8f9a-0061de2cc17b-service-ca-bundle\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626395 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa89d4ee-2ac6-4234-8641-a851e295524d-config\") pod \"service-ca-operator-5bcbc94c78-8j4fl\" (UID: \"aa89d4ee-2ac6-4234-8641-a851e295524d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" Jul 18 00:41:04.626436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626438 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-bound-sa-token\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626477 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626506 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80a66f7f-3a1d-4252-8139-846b74939b67-cert\") pod \"ingress-canary-8n9md\" (UID: \"80a66f7f-3a1d-4252-8139-846b74939b67\") " pod="openshift-ingress-canary/ingress-canary-8n9md" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626531 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzzcp\" (UniqueName: \"kubernetes.io/projected/2f6b75f9-43a8-4d7c-9cd0-7a350a753397-kube-api-access-qzzcp\") pod \"kube-storage-version-migrator-operator-7658d9b749-hdjjz\" (UID: \"2f6b75f9-43a8-4d7c-9cd0-7a350a753397\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626565 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfgkv\" (UniqueName: \"kubernetes.io/projected/7fea5857-2011-457a-91ad-c830f4686d0e-kube-api-access-kfgkv\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626590 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-trusted-ca\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626618 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/602eb3d3-135b-46da-8f9a-0061de2cc17b-tmp\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626643 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/602eb3d3-135b-46da-8f9a-0061de2cc17b-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626686 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztgrf\" (UniqueName: \"kubernetes.io/projected/aa89d4ee-2ac6-4234-8641-a851e295524d-kube-api-access-ztgrf\") pod \"service-ca-operator-5bcbc94c78-8j4fl\" (UID: \"aa89d4ee-2ac6-4234-8641-a851e295524d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626735 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/15ccd479-3004-4964-a298-0c510fc4bf9b-config\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626765 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v7s2\" (UniqueName: \"kubernetes.io/projected/e074f31c-0e2d-4e42-b988-2564e4e2d6a0-kube-api-access-6v7s2\") pod \"volume-data-source-validator-78cf4c6f6-9w9q5\" (UID: \"e074f31c-0e2d-4e42-b988-2564e4e2d6a0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9w9q5" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626794 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7ntf\" (UniqueName: \"kubernetes.io/projected/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-kube-api-access-t7ntf\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626820 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.627119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626858 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f12d587d-fdc5-4173-a8a3-a066a8f8b848-config-volume\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626886 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/602eb3d3-135b-46da-8f9a-0061de2cc17b-serving-cert\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626910 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626938 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vw6m\" (UniqueName: \"kubernetes.io/projected/ecfb0dac-7382-4175-804d-abd202994a71-kube-api-access-8vw6m\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626963 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f6b75f9-43a8-4d7c-9cd0-7a350a753397-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-hdjjz\" (UID: \"2f6b75f9-43a8-4d7c-9cd0-7a350a753397\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.626995 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/602eb3d3-135b-46da-8f9a-0061de2cc17b-snapshots\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.627021 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4pkd\" (UniqueName: \"kubernetes.io/projected/602eb3d3-135b-46da-8f9a-0061de2cc17b-kube-api-access-n4pkd\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.627045 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f12d587d-fdc5-4173-a8a3-a066a8f8b848-tmp-dir\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.627069 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxpxn\" (UniqueName: \"kubernetes.io/projected/f12d587d-fdc5-4173-a8a3-a066a8f8b848-kube-api-access-cxpxn\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.627095 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/602eb3d3-135b-46da-8f9a-0061de2cc17b-tmp\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.627098 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.627160 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15ccd479-3004-4964-a298-0c510fc4bf9b-trusted-ca\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.627317 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/602eb3d3-135b-46da-8f9a-0061de2cc17b-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.627803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.627662 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/602eb3d3-135b-46da-8f9a-0061de2cc17b-snapshots\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.628287 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.627730 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/15ccd479-3004-4964-a298-0c510fc4bf9b-config\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.631260 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.631236 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/602eb3d3-135b-46da-8f9a-0061de2cc17b-serving-cert\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.631389 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.631236 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15ccd479-3004-4964-a298-0c510fc4bf9b-serving-cert\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.636416 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.636363 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn525\" (UniqueName: \"kubernetes.io/projected/15ccd479-3004-4964-a298-0c510fc4bf9b-kube-api-access-rn525\") pod \"console-operator-565b7b569d-kgsmq\" (UID: \"15ccd479-3004-4964-a298-0c510fc4bf9b\") " pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.640886 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.640856 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4pkd\" (UniqueName: \"kubernetes.io/projected/602eb3d3-135b-46da-8f9a-0061de2cc17b-kube-api-access-n4pkd\") pod \"insights-operator-555d9d7885-z987w\" (UID: \"602eb3d3-135b-46da-8f9a-0061de2cc17b\") " pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.641002 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.640893 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfgkv\" (UniqueName: \"kubernetes.io/projected/7fea5857-2011-457a-91ad-c830f4686d0e-kube-api-access-kfgkv\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:04.641002 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.640896 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v7s2\" (UniqueName: \"kubernetes.io/projected/e074f31c-0e2d-4e42-b988-2564e4e2d6a0-kube-api-access-6v7s2\") pod \"volume-data-source-validator-78cf4c6f6-9w9q5\" (UID: \"e074f31c-0e2d-4e42-b988-2564e4e2d6a0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9w9q5" Jul 18 00:41:04.643019 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.642995 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:04.702186 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.702148 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9w9q5" Jul 18 00:41:04.726346 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.726318 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:04.727529 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.727506 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-565xl\" (UniqueName: \"kubernetes.io/projected/444ea3c7-0596-45b0-9db4-7d20da2834a9-kube-api-access-565xl\") pod \"network-check-source-7bc5b6bf85-64kzz\" (UID: \"444ea3c7-0596-45b0-9db4-7d20da2834a9\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-64kzz" Jul 18 00:41:04.727607 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.727543 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/70547afb-1be6-4358-b0cd-5349d7dbf273-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:04.727607 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.727583 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-certificates\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.727744 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.727629 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-default-certificate\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.727799 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.727754 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:04.727868 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.727798 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-image-registry-private-configuration\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.727928 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.727862 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-installation-pull-secrets\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.727928 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.727894 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cltg7\" (UniqueName: \"kubernetes.io/projected/80a66f7f-3a1d-4252-8139-846b74939b67-kube-api-access-cltg7\") pod \"ingress-canary-8n9md\" (UID: \"80a66f7f-3a1d-4252-8139-846b74939b67\") " pod="openshift-ingress-canary/ingress-canary-8n9md" Jul 18 00:41:04.728031 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.727927 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa89d4ee-2ac6-4234-8641-a851e295524d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8j4fl\" (UID: \"aa89d4ee-2ac6-4234-8641-a851e295524d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" Jul 18 00:41:04.728031 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.727957 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9a89e63-02c2-495f-b2d5-72742bc08595-ca-trust-extracted\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.728031 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.727994 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.728176 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728048 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f6b75f9-43a8-4d7c-9cd0-7a350a753397-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-hdjjz\" (UID: \"2f6b75f9-43a8-4d7c-9cd0-7a350a753397\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" Jul 18 00:41:04.728176 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.728096 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 00:41:04.728176 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.728167 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:05.228146972 +0000 UTC m=+36.250281743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : secret "router-metrics-certs-default" not found Jul 18 00:41:04.728176 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.728171 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.728187 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dfcf5fff6-2h49l: secret "image-registry-tls" not found Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.728244 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls podName:a9a89e63-02c2-495f-b2d5-72742bc08595 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:05.228227062 +0000 UTC m=+36.250361846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls") pod "image-registry-5dfcf5fff6-2h49l" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595") : secret "image-registry-tls" not found Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728268 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-certificates\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728304 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/70547afb-1be6-4358-b0cd-5349d7dbf273-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728094 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728448 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb5lr\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-kube-api-access-wb5lr\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728476 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-stats-auth\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728502 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa89d4ee-2ac6-4234-8641-a851e295524d-config\") pod \"service-ca-operator-5bcbc94c78-8j4fl\" (UID: \"aa89d4ee-2ac6-4234-8641-a851e295524d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728533 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-bound-sa-token\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728561 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728586 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80a66f7f-3a1d-4252-8139-846b74939b67-cert\") pod \"ingress-canary-8n9md\" (UID: \"80a66f7f-3a1d-4252-8139-846b74939b67\") " pod="openshift-ingress-canary/ingress-canary-8n9md" Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728597 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:04.733319 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728616 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzzcp\" (UniqueName: \"kubernetes.io/projected/2f6b75f9-43a8-4d7c-9cd0-7a350a753397-kube-api-access-qzzcp\") pod \"kube-storage-version-migrator-operator-7658d9b749-hdjjz\" (UID: \"2f6b75f9-43a8-4d7c-9cd0-7a350a753397\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.728962 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9a89e63-02c2-495f-b2d5-72742bc08595-ca-trust-extracted\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.729130 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729154 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa89d4ee-2ac6-4234-8641-a851e295524d-config\") pod \"service-ca-operator-5bcbc94c78-8j4fl\" (UID: \"aa89d4ee-2ac6-4234-8641-a851e295524d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.729178 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert podName:70547afb-1be6-4358-b0cd-5349d7dbf273 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:05.229162614 +0000 UTC m=+36.251297380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kdk54" (UID: "70547afb-1be6-4358-b0cd-5349d7dbf273") : secret "networking-console-plugin-cert" not found Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729201 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-trusted-ca\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729241 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztgrf\" (UniqueName: \"kubernetes.io/projected/aa89d4ee-2ac6-4234-8641-a851e295524d-kube-api-access-ztgrf\") pod \"service-ca-operator-5bcbc94c78-8j4fl\" (UID: \"aa89d4ee-2ac6-4234-8641-a851e295524d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729280 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7ntf\" (UniqueName: \"kubernetes.io/projected/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-kube-api-access-t7ntf\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729304 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729331 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f12d587d-fdc5-4173-a8a3-a066a8f8b848-config-volume\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729361 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729392 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vw6m\" (UniqueName: \"kubernetes.io/projected/ecfb0dac-7382-4175-804d-abd202994a71-kube-api-access-8vw6m\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729417 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f6b75f9-43a8-4d7c-9cd0-7a350a753397-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-hdjjz\" (UID: \"2f6b75f9-43a8-4d7c-9cd0-7a350a753397\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" Jul 18 00:41:04.733855 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729452 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f12d587d-fdc5-4173-a8a3-a066a8f8b848-tmp-dir\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729478 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxpxn\" (UniqueName: \"kubernetes.io/projected/f12d587d-fdc5-4173-a8a3-a066a8f8b848-kube-api-access-cxpxn\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.729508 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.729623 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.729665 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls podName:bdb58d5a-a4ac-435f-92ae-ff17de63bb0f nodeName:}" failed. No retries permitted until 2026-07-18 00:41:05.229650897 +0000 UTC m=+36.251785667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbdj" (UID: "bdb58d5a-a4ac-435f-92ae-ff17de63bb0f") : secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.730804 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa89d4ee-2ac6-4234-8641-a851e295524d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8j4fl\" (UID: \"aa89d4ee-2ac6-4234-8641-a851e295524d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.730816 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-installation-pull-secrets\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.731087 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-image-registry-private-configuration\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.731108 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-default-certificate\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.731379 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.731422 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls podName:f12d587d-fdc5-4173-a8a3-a066a8f8b848 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:05.231407905 +0000 UTC m=+36.253542669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls") pod "dns-default-2gm4c" (UID: "f12d587d-fdc5-4173-a8a3-a066a8f8b848") : secret "dns-default-metrics-tls" not found Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.731421 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f12d587d-fdc5-4173-a8a3-a066a8f8b848-config-volume\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.734307 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:04.731505 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:05.231492235 +0000 UTC m=+36.253627000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : configmap references non-existent config key: service-ca.crt Jul 18 00:41:04.734736 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.731636 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f6b75f9-43a8-4d7c-9cd0-7a350a753397-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-hdjjz\" (UID: \"2f6b75f9-43a8-4d7c-9cd0-7a350a753397\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" Jul 18 00:41:04.734736 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.731753 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f12d587d-fdc5-4173-a8a3-a066a8f8b848-tmp-dir\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.734736 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.732182 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-trusted-ca\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.734736 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.732411 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-stats-auth\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.734736 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.732417 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f6b75f9-43a8-4d7c-9cd0-7a350a753397-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-hdjjz\" (UID: \"2f6b75f9-43a8-4d7c-9cd0-7a350a753397\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" Jul 18 00:41:04.734736 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.732662 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80a66f7f-3a1d-4252-8139-846b74939b67-cert\") pod \"ingress-canary-8n9md\" (UID: \"80a66f7f-3a1d-4252-8139-846b74939b67\") " pod="openshift-ingress-canary/ingress-canary-8n9md" Jul 18 00:41:04.739403 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.739153 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cltg7\" (UniqueName: \"kubernetes.io/projected/80a66f7f-3a1d-4252-8139-846b74939b67-kube-api-access-cltg7\") pod \"ingress-canary-8n9md\" (UID: \"80a66f7f-3a1d-4252-8139-846b74939b67\") " pod="openshift-ingress-canary/ingress-canary-8n9md" Jul 18 00:41:04.741470 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.741027 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzzcp\" (UniqueName: \"kubernetes.io/projected/2f6b75f9-43a8-4d7c-9cd0-7a350a753397-kube-api-access-qzzcp\") pod \"kube-storage-version-migrator-operator-7658d9b749-hdjjz\" (UID: \"2f6b75f9-43a8-4d7c-9cd0-7a350a753397\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" Jul 18 00:41:04.741470 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.741464 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-bound-sa-token\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.741617 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.741544 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxpxn\" (UniqueName: \"kubernetes.io/projected/f12d587d-fdc5-4173-a8a3-a066a8f8b848-kube-api-access-cxpxn\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:04.741650 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.741635 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vw6m\" (UniqueName: \"kubernetes.io/projected/ecfb0dac-7382-4175-804d-abd202994a71-kube-api-access-8vw6m\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:04.742006 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.741984 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb5lr\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-kube-api-access-wb5lr\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:04.742112 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.742085 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7ntf\" (UniqueName: \"kubernetes.io/projected/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-kube-api-access-t7ntf\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:04.742867 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.742851 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztgrf\" (UniqueName: \"kubernetes.io/projected/aa89d4ee-2ac6-4234-8641-a851e295524d-kube-api-access-ztgrf\") pod \"service-ca-operator-5bcbc94c78-8j4fl\" (UID: \"aa89d4ee-2ac6-4234-8641-a851e295524d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" Jul 18 00:41:04.751311 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.751287 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-z987w" Jul 18 00:41:04.766174 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.766145 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" Jul 18 00:41:04.830162 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.830086 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" Jul 18 00:41:04.830462 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.830425 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-565xl\" (UniqueName: \"kubernetes.io/projected/444ea3c7-0596-45b0-9db4-7d20da2834a9-kube-api-access-565xl\") pod \"network-check-source-7bc5b6bf85-64kzz\" (UID: \"444ea3c7-0596-45b0-9db4-7d20da2834a9\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-64kzz" Jul 18 00:41:04.839350 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.839319 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-565xl\" (UniqueName: \"kubernetes.io/projected/444ea3c7-0596-45b0-9db4-7d20da2834a9-kube-api-access-565xl\") pod \"network-check-source-7bc5b6bf85-64kzz\" (UID: \"444ea3c7-0596-45b0-9db4-7d20da2834a9\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-64kzz" Jul 18 00:41:04.854863 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.854813 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8n9md" Jul 18 00:41:04.867497 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:04.867475 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-64kzz" Jul 18 00:41:05.133505 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.133413 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:05.133659 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.133576 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 00:41:05.133659 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.133649 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls podName:7fea5857-2011-457a-91ad-c830f4686d0e nodeName:}" failed. No retries permitted until 2026-07-18 00:41:06.133631208 +0000 UTC m=+37.155765986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-67bw6" (UID: "7fea5857-2011-457a-91ad-c830f4686d0e") : secret "samples-operator-tls" not found Jul 18 00:41:05.234052 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.234018 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:05.234052 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.234056 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:05.234311 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.234081 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:05.234311 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.234126 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:05.234311 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.234150 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:05.234311 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.234179 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:05.234311 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234196 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 00:41:05.234311 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234217 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dfcf5fff6-2h49l: secret "image-registry-tls" not found Jul 18 00:41:05.234311 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234193 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 00:41:05.234311 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234278 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 00:41:05.234311 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234293 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 00:41:05.234676 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234280 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls podName:a9a89e63-02c2-495f-b2d5-72742bc08595 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:06.234261694 +0000 UTC m=+37.256396460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls") pod "image-registry-5dfcf5fff6-2h49l" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595") : secret "image-registry-tls" not found Jul 18 00:41:05.234676 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234388 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:06.234369635 +0000 UTC m=+37.256504407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : configmap references non-existent config key: service-ca.crt Jul 18 00:41:05.234676 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234277 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:05.234676 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234474 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:06.234463623 +0000 UTC m=+37.256598384 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : secret "router-metrics-certs-default" not found Jul 18 00:41:05.234676 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234490 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert podName:70547afb-1be6-4358-b0cd-5349d7dbf273 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:06.234481752 +0000 UTC m=+37.256616520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kdk54" (UID: "70547afb-1be6-4358-b0cd-5349d7dbf273") : secret "networking-console-plugin-cert" not found Jul 18 00:41:05.234676 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234510 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls podName:f12d587d-fdc5-4173-a8a3-a066a8f8b848 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:06.234502825 +0000 UTC m=+37.256637592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls") pod "dns-default-2gm4c" (UID: "f12d587d-fdc5-4173-a8a3-a066a8f8b848") : secret "dns-default-metrics-tls" not found Jul 18 00:41:05.234676 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:05.234528 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls podName:bdb58d5a-a4ac-435f-92ae-ff17de63bb0f nodeName:}" failed. No retries permitted until 2026-07-18 00:41:06.234520227 +0000 UTC m=+37.256654991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbdj" (UID: "bdb58d5a-a4ac-435f-92ae-ff17de63bb0f") : secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:05.451529 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.451446 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:41:05.451529 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.451490 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:41:05.451529 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.451489 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:41:05.454017 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.453993 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2stqj\"" Jul 18 00:41:05.454172 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.454050 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 00:41:05.454275 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.454261 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 00:41:05.454384 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.454367 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9tdp4\"" Jul 18 00:41:05.686361 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.686328 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:05.736213 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:05.736134 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:06.148586 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.148227 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:06.148586 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.148388 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 00:41:06.148586 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.148584 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls podName:7fea5857-2011-457a-91ad-c830f4686d0e nodeName:}" failed. No retries permitted until 2026-07-18 00:41:08.14856656 +0000 UTC m=+39.170701339 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-67bw6" (UID: "7fea5857-2011-457a-91ad-c830f4686d0e") : secret "samples-operator-tls" not found Jul 18 00:41:06.149928 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.149905 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8n9md"] Jul 18 00:41:06.151165 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.151139 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-64kzz"] Jul 18 00:41:06.154900 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:06.154807 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod444ea3c7_0596_45b0_9db4_7d20da2834a9.slice/crio-5a172aff12185432c4be72a370c17c8b6bb2bb050605da43b0d6000cc35f09e8 WatchSource:0}: Error finding container 5a172aff12185432c4be72a370c17c8b6bb2bb050605da43b0d6000cc35f09e8: Status 404 returned error can't find the container with id 5a172aff12185432c4be72a370c17c8b6bb2bb050605da43b0d6000cc35f09e8 Jul 18 00:41:06.155466 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:06.155434 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a66f7f_3a1d_4252_8139_846b74939b67.slice/crio-e5d384756839495e9ff637e113280a58936e2542a3c8a26e2272abd9ebce9f0f WatchSource:0}: Error finding container e5d384756839495e9ff637e113280a58936e2542a3c8a26e2272abd9ebce9f0f: Status 404 returned error can't find the container with id e5d384756839495e9ff637e113280a58936e2542a3c8a26e2272abd9ebce9f0f Jul 18 00:41:06.164361 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.164334 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-kgsmq"] Jul 18 00:41:06.165146 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.165085 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl"] Jul 18 00:41:06.166266 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.166209 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-z987w"] Jul 18 00:41:06.172545 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:06.172520 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa89d4ee_2ac6_4234_8641_a851e295524d.slice/crio-9b34962ed2b3d66242587db75179e37fd6224d44962878bc0acff58a71faea5a WatchSource:0}: Error finding container 9b34962ed2b3d66242587db75179e37fd6224d44962878bc0acff58a71faea5a: Status 404 returned error can't find the container with id 9b34962ed2b3d66242587db75179e37fd6224d44962878bc0acff58a71faea5a Jul 18 00:41:06.173185 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:06.173153 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ccd479_3004_4964_a298_0c510fc4bf9b.slice/crio-659214d0dc5c26509b0d28e6d75e20cf10d21de250d7dbd141e263802f665d69 WatchSource:0}: Error finding container 659214d0dc5c26509b0d28e6d75e20cf10d21de250d7dbd141e263802f665d69: Status 404 returned error can't find the container with id 659214d0dc5c26509b0d28e6d75e20cf10d21de250d7dbd141e263802f665d69 Jul 18 00:41:06.173997 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:06.173822 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod602eb3d3_135b_46da_8f9a_0061de2cc17b.slice/crio-be783b322611d62ecc13c35faa87eb8a88f53b9ee415d6fecff56b8e99a27400 WatchSource:0}: Error finding container be783b322611d62ecc13c35faa87eb8a88f53b9ee415d6fecff56b8e99a27400: Status 404 returned error can't find the container with id be783b322611d62ecc13c35faa87eb8a88f53b9ee415d6fecff56b8e99a27400 Jul 18 00:41:06.176106 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.176082 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9w9q5"] Jul 18 00:41:06.179005 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:06.178979 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode074f31c_0e2d_4e42_b988_2564e4e2d6a0.slice/crio-b122a86ae16b71c58d73d1c9176906b314da071c5c03d70cc3c26d495b5d7f87 WatchSource:0}: Error finding container b122a86ae16b71c58d73d1c9176906b314da071c5c03d70cc3c26d495b5d7f87: Status 404 returned error can't find the container with id b122a86ae16b71c58d73d1c9176906b314da071c5c03d70cc3c26d495b5d7f87 Jul 18 00:41:06.180258 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.180237 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz"] Jul 18 00:41:06.182680 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:06.182619 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6b75f9_43a8_4d7c_9cd0_7a350a753397.slice/crio-19ed2f5d14a2b62150e04d10a36808aa44355384d0d9a655193e0f00128f4d07 WatchSource:0}: Error finding container 19ed2f5d14a2b62150e04d10a36808aa44355384d0d9a655193e0f00128f4d07: Status 404 returned error can't find the container with id 19ed2f5d14a2b62150e04d10a36808aa44355384d0d9a655193e0f00128f4d07 Jul 18 00:41:06.249477 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.249449 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:06.249622 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.249487 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:06.249622 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.249507 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:06.249622 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.249537 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:06.249622 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.249556 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:06.249622 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.249591 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:06.249622 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249609 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 00:41:06.249881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249626 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 00:41:06.249881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249664 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 00:41:06.249881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249680 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:08.249665413 +0000 UTC m=+39.271800180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : secret "router-metrics-certs-default" not found Jul 18 00:41:06.249881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249695 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert podName:70547afb-1be6-4358-b0cd-5349d7dbf273 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:08.249688211 +0000 UTC m=+39.271822973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kdk54" (UID: "70547afb-1be6-4358-b0cd-5349d7dbf273") : secret "networking-console-plugin-cert" not found Jul 18 00:41:06.249881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249724 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls podName:f12d587d-fdc5-4173-a8a3-a066a8f8b848 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:08.249702558 +0000 UTC m=+39.271837320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls") pod "dns-default-2gm4c" (UID: "f12d587d-fdc5-4173-a8a3-a066a8f8b848") : secret "dns-default-metrics-tls" not found Jul 18 00:41:06.249881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249762 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 00:41:06.249881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249769 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:08.249752278 +0000 UTC m=+39.271887056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : configmap references non-existent config key: service-ca.crt Jul 18 00:41:06.249881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249772 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dfcf5fff6-2h49l: secret "image-registry-tls" not found Jul 18 00:41:06.249881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249807 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls podName:a9a89e63-02c2-495f-b2d5-72742bc08595 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:08.249799221 +0000 UTC m=+39.271933982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls") pod "image-registry-5dfcf5fff6-2h49l" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595") : secret "image-registry-tls" not found Jul 18 00:41:06.249881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249771 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:06.249881 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:06.249861 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls podName:bdb58d5a-a4ac-435f-92ae-ff17de63bb0f nodeName:}" failed. No retries permitted until 2026-07-18 00:41:08.249852659 +0000 UTC m=+39.271987424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbdj" (UID: "bdb58d5a-a4ac-435f-92ae-ff17de63bb0f") : secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:06.537895 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.537801 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:06.612360 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.612226 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:06.712131 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.712052 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:06.743070 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.742965 2585 generic.go:356] "Generic (PLEG): container finished" podID="513eeb46-6cbf-4b38-a4a6-e9fcc1f93767" containerID="79043084bb00a2237c8ce115348c7d8c84a8603ca0cf9f7992c34ac74d49525c" exitCode=0 Jul 18 00:41:06.743070 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.743055 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p729n" event={"ID":"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767","Type":"ContainerDied","Data":"79043084bb00a2237c8ce115348c7d8c84a8603ca0cf9f7992c34ac74d49525c"} Jul 18 00:41:06.747503 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.747425 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8n9md" event={"ID":"80a66f7f-3a1d-4252-8139-846b74939b67","Type":"ContainerStarted","Data":"e5d384756839495e9ff637e113280a58936e2542a3c8a26e2272abd9ebce9f0f"} Jul 18 00:41:06.751383 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.751294 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9w9q5" event={"ID":"e074f31c-0e2d-4e42-b988-2564e4e2d6a0","Type":"ContainerStarted","Data":"b122a86ae16b71c58d73d1c9176906b314da071c5c03d70cc3c26d495b5d7f87"} Jul 18 00:41:06.753274 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.753206 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-z987w" event={"ID":"602eb3d3-135b-46da-8f9a-0061de2cc17b","Type":"ContainerStarted","Data":"be783b322611d62ecc13c35faa87eb8a88f53b9ee415d6fecff56b8e99a27400"} Jul 18 00:41:06.758653 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.758627 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-64kzz" event={"ID":"444ea3c7-0596-45b0-9db4-7d20da2834a9","Type":"ContainerStarted","Data":"5a172aff12185432c4be72a370c17c8b6bb2bb050605da43b0d6000cc35f09e8"} Jul 18 00:41:06.766079 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.766024 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" event={"ID":"2f6b75f9-43a8-4d7c-9cd0-7a350a753397","Type":"ContainerStarted","Data":"19ed2f5d14a2b62150e04d10a36808aa44355384d0d9a655193e0f00128f4d07"} Jul 18 00:41:06.771316 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.771292 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" event={"ID":"aa89d4ee-2ac6-4234-8641-a851e295524d","Type":"ContainerStarted","Data":"9b34962ed2b3d66242587db75179e37fd6224d44962878bc0acff58a71faea5a"} Jul 18 00:41:06.773096 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:06.773067 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" event={"ID":"15ccd479-3004-4964-a298-0c510fc4bf9b","Type":"ContainerStarted","Data":"659214d0dc5c26509b0d28e6d75e20cf10d21de250d7dbd141e263802f665d69"} Jul 18 00:41:07.784131 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:07.783237 2585 generic.go:356] "Generic (PLEG): container finished" podID="513eeb46-6cbf-4b38-a4a6-e9fcc1f93767" containerID="773100a032e3f2ab0a872cd18f1eccd21a0c73236b540aee25c1a370b3ee1e3c" exitCode=0 Jul 18 00:41:07.784131 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:07.783288 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p729n" event={"ID":"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767","Type":"ContainerDied","Data":"773100a032e3f2ab0a872cd18f1eccd21a0c73236b540aee25c1a370b3ee1e3c"} Jul 18 00:41:08.169367 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.168423 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:08.169367 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.168807 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 00:41:08.169367 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.168966 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls podName:7fea5857-2011-457a-91ad-c830f4686d0e nodeName:}" failed. No retries permitted until 2026-07-18 00:41:12.168942683 +0000 UTC m=+43.191077449 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-67bw6" (UID: "7fea5857-2011-457a-91ad-c830f4686d0e") : secret "samples-operator-tls" not found Jul 18 00:41:08.269535 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.269496 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:08.269671 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.269651 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:08.269735 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.269694 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:08.269786 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.269729 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:08.269786 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.269779 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:08.269907 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.269812 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:08.269988 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.269971 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:12.26995051 +0000 UTC m=+43.292085275 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : configmap references non-existent config key: service-ca.crt Jul 18 00:41:08.270094 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.270078 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:08.270164 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.270125 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls podName:bdb58d5a-a4ac-435f-92ae-ff17de63bb0f nodeName:}" failed. No retries permitted until 2026-07-18 00:41:12.270111978 +0000 UTC m=+43.292246747 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbdj" (UID: "bdb58d5a-a4ac-435f-92ae-ff17de63bb0f") : secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:08.270223 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.270184 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 00:41:08.270223 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.270216 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:12.27020616 +0000 UTC m=+43.292340923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : secret "router-metrics-certs-default" not found Jul 18 00:41:08.270323 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.270279 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 00:41:08.270323 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.270290 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dfcf5fff6-2h49l: secret "image-registry-tls" not found Jul 18 00:41:08.270323 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.270319 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls podName:a9a89e63-02c2-495f-b2d5-72742bc08595 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:12.270308701 +0000 UTC m=+43.292443468 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls") pod "image-registry-5dfcf5fff6-2h49l" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595") : secret "image-registry-tls" not found Jul 18 00:41:08.270466 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.270370 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 00:41:08.270466 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.270404 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert podName:70547afb-1be6-4358-b0cd-5349d7dbf273 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:12.270392556 +0000 UTC m=+43.292527323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kdk54" (UID: "70547afb-1be6-4358-b0cd-5349d7dbf273") : secret "networking-console-plugin-cert" not found Jul 18 00:41:08.270466 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.270454 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 00:41:08.270608 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:08.270481 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls podName:f12d587d-fdc5-4173-a8a3-a066a8f8b848 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:12.270471787 +0000 UTC m=+43.292606550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls") pod "dns-default-2gm4c" (UID: "f12d587d-fdc5-4173-a8a3-a066a8f8b848") : secret "dns-default-metrics-tls" not found Jul 18 00:41:08.614969 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.614700 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:08.714776 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.714728 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:08.792374 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.792084 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p729n" event={"ID":"513eeb46-6cbf-4b38-a4a6-e9fcc1f93767","Type":"ContainerStarted","Data":"a3dcd5c7ac4400eba33b01d6bffd2290f56b18aebd8a69095845bd3c24f345f8"} Jul 18 00:41:08.794910 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.794868 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9w9q5" event={"ID":"e074f31c-0e2d-4e42-b988-2564e4e2d6a0","Type":"ContainerStarted","Data":"79ecdcc92a5c4fcb9e63456ba68224d9bef99bdaa44335225e2f18e157e054b5"} Jul 18 00:41:08.815418 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.814222 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p729n" podStartSLOduration=15.010043175 podStartE2EDuration="38.814200861s" totalImagesPullingTime="23.804157686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:08.812736359 +0000 UTC m=+39.834871156" watchObservedRunningTime="2026-07-18 00:41:08.814200861 +0000 UTC m=+39.836335645" Jul 18 00:41:08.826996 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.826920 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9w9q5" podStartSLOduration=26.883358591 podStartE2EDuration="28.826899797s" totalImagesPullingTime="1.943541206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:08.826544133 +0000 UTC m=+39.848678928" watchObservedRunningTime="2026-07-18 00:41:08.826899797 +0000 UTC m=+39.849034634" Jul 18 00:41:08.909812 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.909720 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:08.968065 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:08.967946 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:09.023811 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:09.023777 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:09.073440 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:09.073407 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:09.497660 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:09.497624 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:09.550431 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:09.550386 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:09.629032 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:09.628936 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:09.924416 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:09.924381 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:09.978000 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:09.977971 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:10.046321 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:10.046290 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:12.207324 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:12.207285 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:12.207816 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.207445 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 00:41:12.207816 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.207517 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls podName:7fea5857-2011-457a-91ad-c830f4686d0e nodeName:}" failed. No retries permitted until 2026-07-18 00:41:20.207499181 +0000 UTC m=+51.229633944 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-67bw6" (UID: "7fea5857-2011-457a-91ad-c830f4686d0e") : secret "samples-operator-tls" not found Jul 18 00:41:12.307769 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:12.307735 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:12.307769 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:12.307776 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:12.308004 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:12.307798 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:12.308004 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:12.307843 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:12.308004 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:12.307870 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:12.308004 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:12.307895 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:12.308004 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.307904 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 00:41:12.308004 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.307933 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 00:41:12.308004 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.307959 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 00:41:12.308004 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.307979 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:20.307957827 +0000 UTC m=+51.330092588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : secret "router-metrics-certs-default" not found Jul 18 00:41:12.308004 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.307907 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 00:41:12.308320 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.308016 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dfcf5fff6-2h49l: secret "image-registry-tls" not found Jul 18 00:41:12.308320 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.308004 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls podName:f12d587d-fdc5-4173-a8a3-a066a8f8b848 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:20.307986239 +0000 UTC m=+51.330121009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls") pod "dns-default-2gm4c" (UID: "f12d587d-fdc5-4173-a8a3-a066a8f8b848") : secret "dns-default-metrics-tls" not found Jul 18 00:41:12.308320 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.308039 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:12.308320 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.308053 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert podName:70547afb-1be6-4358-b0cd-5349d7dbf273 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:20.30802912 +0000 UTC m=+51.330163886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kdk54" (UID: "70547afb-1be6-4358-b0cd-5349d7dbf273") : secret "networking-console-plugin-cert" not found Jul 18 00:41:12.308320 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.308082 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:20.308072552 +0000 UTC m=+51.330207330 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : configmap references non-existent config key: service-ca.crt Jul 18 00:41:12.308320 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.308103 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls podName:a9a89e63-02c2-495f-b2d5-72742bc08595 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:20.308093591 +0000 UTC m=+51.330228360 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls") pod "image-registry-5dfcf5fff6-2h49l" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595") : secret "image-registry-tls" not found Jul 18 00:41:12.308320 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:12.308121 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls podName:bdb58d5a-a4ac-435f-92ae-ff17de63bb0f nodeName:}" failed. No retries permitted until 2026-07-18 00:41:20.30811336 +0000 UTC m=+51.330248127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbdj" (UID: "bdb58d5a-a4ac-435f-92ae-ff17de63bb0f") : secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:14.816229 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.816197 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" event={"ID":"aa89d4ee-2ac6-4234-8641-a851e295524d","Type":"ContainerStarted","Data":"3b089f7ba51f336dbdd987245694f5ee823aeee9a20019bf26d76b3e01c4ec70"} Jul 18 00:41:14.818002 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.817984 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/0.log" Jul 18 00:41:14.818130 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.818018 2585 generic.go:356] "Generic (PLEG): container finished" podID="15ccd479-3004-4964-a298-0c510fc4bf9b" containerID="a82245450c969142aa3f5b56843591be16869b0591c5048ccbdcd78efe8368f6" exitCode=255 Jul 18 00:41:14.818191 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.818118 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" event={"ID":"15ccd479-3004-4964-a298-0c510fc4bf9b","Type":"ContainerDied","Data":"a82245450c969142aa3f5b56843591be16869b0591c5048ccbdcd78efe8368f6"} Jul 18 00:41:14.818299 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.818283 2585 scope.go:117] "RemoveContainer" containerID="a82245450c969142aa3f5b56843591be16869b0591c5048ccbdcd78efe8368f6" Jul 18 00:41:14.820051 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.820026 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8n9md" event={"ID":"80a66f7f-3a1d-4252-8139-846b74939b67","Type":"ContainerStarted","Data":"3ff55553c1a1cd2fb83fd1e560998d49fe39946ea39d9b0a334b3203bf11edbe"} Jul 18 00:41:14.822055 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.822034 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-z987w" event={"ID":"602eb3d3-135b-46da-8f9a-0061de2cc17b","Type":"ContainerStarted","Data":"932c452f132cddcca7f986e6cc494dacd6d001dfed511c7510ba6debf0f39abf"} Jul 18 00:41:14.823687 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.823643 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-64kzz" event={"ID":"444ea3c7-0596-45b0-9db4-7d20da2834a9","Type":"ContainerStarted","Data":"e327713e3b15735d87344ecac66ca230e499cf97271830c5725cf5282a359170"} Jul 18 00:41:14.825063 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.825039 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" event={"ID":"2f6b75f9-43a8-4d7c-9cd0-7a350a753397","Type":"ContainerStarted","Data":"81a4956c394d6823bc7dc2d9fe239a9686ebe9d1d068fb616ee8c19d1224e03e"} Jul 18 00:41:14.831951 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.831913 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" podStartSLOduration=27.238009047 podStartE2EDuration="34.831901757s" totalImagesPullingTime="7.59389271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:14.830746091 +0000 UTC m=+45.852880878" watchObservedRunningTime="2026-07-18 00:41:14.831901757 +0000 UTC m=+45.854036540" Jul 18 00:41:14.850096 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.849860 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" podStartSLOduration=27.265732603 podStartE2EDuration="34.84981251s" totalImagesPullingTime="7.584079907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:14.847921031 +0000 UTC m=+45.870055815" watchObservedRunningTime="2026-07-18 00:41:14.84981251 +0000 UTC m=+45.871947297" Jul 18 00:41:14.858633 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.858604 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:14.865991 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.865930 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8n9md" podStartSLOduration=3.254265504 podStartE2EDuration="10.865913307s" totalImagesPullingTime="7.611647803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:14.865217141 +0000 UTC m=+45.887351938" watchObservedRunningTime="2026-07-18 00:41:14.865913307 +0000 UTC m=+45.888048092" Jul 18 00:41:14.880690 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.880617 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-64kzz" podStartSLOduration=27.258632046 podStartE2EDuration="34.880597403s" totalImagesPullingTime="7.621965357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:14.880188839 +0000 UTC m=+45.902323624" watchObservedRunningTime="2026-07-18 00:41:14.880597403 +0000 UTC m=+45.902732189" Jul 18 00:41:14.912183 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.912128 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-z987w" podStartSLOduration=27.318998238 podStartE2EDuration="34.912110359s" totalImagesPullingTime="7.593112121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:14.91094077 +0000 UTC m=+45.933075555" watchObservedRunningTime="2026-07-18 00:41:14.912110359 +0000 UTC m=+45.934245143" Jul 18 00:41:14.924380 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:14.924348 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:15.301987 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.301891 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:15.349646 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.349599 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:15.577955 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.577870 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:15.664662 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.664624 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:15.764994 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.764960 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:15.834423 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.834341 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 00:41:15.834847 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.834793 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/0.log" Jul 18 00:41:15.834910 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.834849 2585 generic.go:356] "Generic (PLEG): container finished" podID="15ccd479-3004-4964-a298-0c510fc4bf9b" containerID="848edffb52bdbd8dadbbbea877851ec2130b6803e1195d1b9bd8564a0193245f" exitCode=255 Jul 18 00:41:15.834910 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.834876 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" event={"ID":"15ccd479-3004-4964-a298-0c510fc4bf9b","Type":"ContainerDied","Data":"848edffb52bdbd8dadbbbea877851ec2130b6803e1195d1b9bd8564a0193245f"} Jul 18 00:41:15.834989 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.834963 2585 scope.go:117] "RemoveContainer" containerID="a82245450c969142aa3f5b56843591be16869b0591c5048ccbdcd78efe8368f6" Jul 18 00:41:15.835188 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.835168 2585 scope.go:117] "RemoveContainer" containerID="848edffb52bdbd8dadbbbea877851ec2130b6803e1195d1b9bd8564a0193245f" Jul 18 00:41:15.835386 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:15.835356 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-kgsmq_openshift-console-operator(15ccd479-3004-4964-a298-0c510fc4bf9b)\"" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" podUID="15ccd479-3004-4964-a298-0c510fc4bf9b" Jul 18 00:41:15.952769 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.952723 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-f2qcd"] Jul 18 00:41:15.977287 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.977250 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f2qcd"] Jul 18 00:41:15.977465 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.977367 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:15.979614 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.979589 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 18 00:41:15.979759 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.979589 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 18 00:41:15.980036 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.980018 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cqrgt\"" Jul 18 00:41:15.983034 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:15.983009 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:16.041934 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.041902 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc428557-f751-4d98-abc8-7d1e12a09593-crio-socket\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.042115 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.041948 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc428557-f751-4d98-abc8-7d1e12a09593-data-volume\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.042115 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.042049 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.042223 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.042134 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.042297 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.042277 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htd68\" (UniqueName: \"kubernetes.io/projected/cc428557-f751-4d98-abc8-7d1e12a09593-kube-api-access-htd68\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.052067 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.052033 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:16.143006 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.142916 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc428557-f751-4d98-abc8-7d1e12a09593-crio-socket\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.143006 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.142964 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc428557-f751-4d98-abc8-7d1e12a09593-data-volume\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.143185 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.143079 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.143185 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.143156 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc428557-f751-4d98-abc8-7d1e12a09593-crio-socket\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.143185 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.143174 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.143312 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:16.143186 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 00:41:16.143312 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:16.143258 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm podName:cc428557-f751-4d98-abc8-7d1e12a09593 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:16.643235464 +0000 UTC m=+47.665370240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f2qcd" (UID: "cc428557-f751-4d98-abc8-7d1e12a09593") : configmap "kube-rbac-proxy" not found Jul 18 00:41:16.143312 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.143263 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc428557-f751-4d98-abc8-7d1e12a09593-data-volume\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.143472 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:16.143323 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 00:41:16.143472 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.143341 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htd68\" (UniqueName: \"kubernetes.io/projected/cc428557-f751-4d98-abc8-7d1e12a09593-kube-api-access-htd68\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.143472 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:16.143378 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls podName:cc428557-f751-4d98-abc8-7d1e12a09593 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:16.643362478 +0000 UTC m=+47.665497257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f2qcd" (UID: "cc428557-f751-4d98-abc8-7d1e12a09593") : secret "insights-runtime-extractor-tls" not found Jul 18 00:41:16.157105 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.157078 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htd68\" (UniqueName: \"kubernetes.io/projected/cc428557-f751-4d98-abc8-7d1e12a09593-kube-api-access-htd68\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.648635 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.648596 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.648825 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.648738 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:16.648825 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:16.648768 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 00:41:16.648825 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:16.648812 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 00:41:16.648957 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:16.648858 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls podName:cc428557-f751-4d98-abc8-7d1e12a09593 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:17.648822765 +0000 UTC m=+48.670957530 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f2qcd" (UID: "cc428557-f751-4d98-abc8-7d1e12a09593") : secret "insights-runtime-extractor-tls" not found Jul 18 00:41:16.648957 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:16.648874 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm podName:cc428557-f751-4d98-abc8-7d1e12a09593 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:17.648867003 +0000 UTC m=+48.671001765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f2qcd" (UID: "cc428557-f751-4d98-abc8-7d1e12a09593") : configmap "kube-rbac-proxy" not found Jul 18 00:41:16.840309 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.840282 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 00:41:16.840690 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:16.840591 2585 scope.go:117] "RemoveContainer" containerID="848edffb52bdbd8dadbbbea877851ec2130b6803e1195d1b9bd8564a0193245f" Jul 18 00:41:16.840761 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:16.840742 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-kgsmq_openshift-console-operator(15ccd479-3004-4964-a298-0c510fc4bf9b)\"" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" podUID="15ccd479-3004-4964-a298-0c510fc4bf9b" Jul 18 00:41:17.192975 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:17.192937 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:17.253168 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:17.253131 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:17.656750 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:17.656718 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:17.656938 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:17.656878 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:17.656938 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:17.656905 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 00:41:17.657021 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:17.656954 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 00:41:17.657021 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:17.656978 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls podName:cc428557-f751-4d98-abc8-7d1e12a09593 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:19.656962983 +0000 UTC m=+50.679097744 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f2qcd" (UID: "cc428557-f751-4d98-abc8-7d1e12a09593") : secret "insights-runtime-extractor-tls" not found Jul 18 00:41:17.657021 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:17.656996 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm podName:cc428557-f751-4d98-abc8-7d1e12a09593 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:19.656984607 +0000 UTC m=+50.679119368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f2qcd" (UID: "cc428557-f751-4d98-abc8-7d1e12a09593") : configmap "kube-rbac-proxy" not found Jul 18 00:41:18.210882 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:18.210850 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:18.355195 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:18.355168 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mfx87_4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3/dns-node-resolver/0.log" Jul 18 00:41:19.156788 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:19.156761 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cxmc9_d800e91f-1436-4859-83e2-9bd813856736/node-ca/0.log" Jul 18 00:41:19.675471 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:19.675436 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:19.675876 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:19.675504 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:19.675876 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:19.675557 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 00:41:19.675876 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:19.675642 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm podName:cc428557-f751-4d98-abc8-7d1e12a09593 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:23.675622975 +0000 UTC m=+54.697757739 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f2qcd" (UID: "cc428557-f751-4d98-abc8-7d1e12a09593") : configmap "kube-rbac-proxy" not found Jul 18 00:41:19.675876 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:19.675662 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 00:41:19.675876 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:19.675728 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls podName:cc428557-f751-4d98-abc8-7d1e12a09593 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:23.675713756 +0000 UTC m=+54.697848518 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f2qcd" (UID: "cc428557-f751-4d98-abc8-7d1e12a09593") : secret "insights-runtime-extractor-tls" not found Jul 18 00:41:19.954486 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:19.954402 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8n9md_80a66f7f-3a1d-4252-8139-846b74939b67/serve-healthcheck-canary/0.log" Jul 18 00:41:20.282233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:20.282141 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:20.282396 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.282319 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 00:41:20.282451 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.282403 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls podName:7fea5857-2011-457a-91ad-c830f4686d0e nodeName:}" failed. No retries permitted until 2026-07-18 00:41:36.282381427 +0000 UTC m=+67.304516203 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-67bw6" (UID: "7fea5857-2011-457a-91ad-c830f4686d0e") : secret "samples-operator-tls" not found Jul 18 00:41:20.383677 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:20.383634 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:20.383677 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:20.383682 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:20.383981 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:20.383714 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:20.383981 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:20.383756 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:20.383981 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.383780 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 00:41:20.383981 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.383864 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 00:41:20.383981 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.383867 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 00:41:20.383981 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.383879 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 00:41:20.383981 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.383887 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dfcf5fff6-2h49l: secret "image-registry-tls" not found Jul 18 00:41:20.383981 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.383865 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:36.383848721 +0000 UTC m=+67.405983483 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : secret "router-metrics-certs-default" not found Jul 18 00:41:20.383981 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:20.383960 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:20.384385 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:20.383998 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:20.384385 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.384043 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls podName:f12d587d-fdc5-4173-a8a3-a066a8f8b848 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:36.384025917 +0000 UTC m=+67.406160681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls") pod "dns-default-2gm4c" (UID: "f12d587d-fdc5-4173-a8a3-a066a8f8b848") : secret "dns-default-metrics-tls" not found Jul 18 00:41:20.384385 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.384067 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle podName:ecfb0dac-7382-4175-804d-abd202994a71 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:36.384051434 +0000 UTC m=+67.406186207 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle") pod "router-default-559bddd8f8-w24db" (UID: "ecfb0dac-7382-4175-804d-abd202994a71") : configmap references non-existent config key: service-ca.crt Jul 18 00:41:20.384385 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.384070 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:20.384385 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.384095 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert podName:70547afb-1be6-4358-b0cd-5349d7dbf273 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:36.384086287 +0000 UTC m=+67.406221061 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kdk54" (UID: "70547afb-1be6-4358-b0cd-5349d7dbf273") : secret "networking-console-plugin-cert" not found Jul 18 00:41:20.384385 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.384114 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls podName:a9a89e63-02c2-495f-b2d5-72742bc08595 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:36.384104213 +0000 UTC m=+67.406238980 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls") pod "image-registry-5dfcf5fff6-2h49l" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595") : secret "image-registry-tls" not found Jul 18 00:41:20.384385 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:20.384137 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls podName:bdb58d5a-a4ac-435f-92ae-ff17de63bb0f nodeName:}" failed. No retries permitted until 2026-07-18 00:41:36.384122184 +0000 UTC m=+67.406256948 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbdj" (UID: "bdb58d5a-a4ac-435f-92ae-ff17de63bb0f") : secret "cluster-monitoring-operator-tls" not found Jul 18 00:41:20.756693 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:20.756664 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-hdjjz_2f6b75f9-43a8-4d7c-9cd0-7a350a753397/kube-storage-version-migrator-operator/0.log" Jul 18 00:41:21.052736 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:21.052636 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:21.152000 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:21.151962 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:21.252215 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:21.252180 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:21.354064 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:21.353956 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:21.781600 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:21.781563 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:21.882792 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:21.882755 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:22.135570 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:22.135472 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:23.138456 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:23.138418 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:23.330032 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:23.329999 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:23.712812 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:23.712775 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:23.713058 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:23.712877 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:23.713058 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:23.713017 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 00:41:23.713153 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:23.713092 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls podName:cc428557-f751-4d98-abc8-7d1e12a09593 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:31.713076857 +0000 UTC m=+62.735211619 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f2qcd" (UID: "cc428557-f751-4d98-abc8-7d1e12a09593") : secret "insights-runtime-extractor-tls" not found Jul 18 00:41:23.713358 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:23.713341 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc428557-f751-4d98-abc8-7d1e12a09593-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:23.736660 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:23.736587 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:23.932444 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:23.932405 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:24.033457 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:24.033369 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:24.085231 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:24.085199 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:24.278098 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:24.278062 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:24.379260 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:24.379170 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:24.426503 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:24.426471 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:24.726792 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:24.726759 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:24.726993 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:24.726888 2585 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:24.727259 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:24.727245 2585 scope.go:117] "RemoveContainer" containerID="848edffb52bdbd8dadbbbea877851ec2130b6803e1195d1b9bd8564a0193245f" Jul 18 00:41:24.727417 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:24.727397 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-kgsmq_openshift-console-operator(15ccd479-3004-4964-a298-0c510fc4bf9b)\"" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" podUID="15ccd479-3004-4964-a298-0c510fc4bf9b" Jul 18 00:41:24.738186 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:24.738136 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:24.940496 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:24.940458 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:24.998854 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:24.998753 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:25.536263 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:25.536227 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:25.729523 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:25.729486 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:25.786123 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:25.786095 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:26.132508 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:26.132468 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:26.336687 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:26.336653 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:26.437094 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:26.437065 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:26.538640 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:26.538608 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:26.922342 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:26.922278 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:26.973908 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:26.973871 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:27.331971 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:27.331888 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:27.730101 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:27.730067 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:27.929323 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:27.929289 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:28.332482 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:28.332453 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:29.134298 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:29.134262 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:29.531054 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:29.531014 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:29.728681 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:29.728643 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:29.934635 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:29.934599 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:30.139555 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:30.139519 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:31.136447 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:31.136415 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:31.344151 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:31.344108 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:31.537573 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:31.537536 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:31.735885 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:31.735824 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:31.736071 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:31.736032 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c6t5d" Jul 18 00:41:31.788130 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:31.788039 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:31.790431 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:31.790397 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc428557-f751-4d98-abc8-7d1e12a09593-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f2qcd\" (UID: \"cc428557-f751-4d98-abc8-7d1e12a09593\") " pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:31.889259 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:31.889234 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cqrgt\"" Jul 18 00:41:31.897300 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:31.897280 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f2qcd" Jul 18 00:41:32.022563 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:32.022526 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f2qcd"] Jul 18 00:41:32.025417 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:32.025393 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc428557_f751_4d98_abc8_7d1e12a09593.slice/crio-166b4430f2a0f0d8f03be5abb884ec25eba640549785bfa674beb1e6cc9eea18 WatchSource:0}: Error finding container 166b4430f2a0f0d8f03be5abb884ec25eba640549785bfa674beb1e6cc9eea18: Status 404 returned error can't find the container with id 166b4430f2a0f0d8f03be5abb884ec25eba640549785bfa674beb1e6cc9eea18 Jul 18 00:41:32.895102 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:32.895053 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f2qcd" event={"ID":"cc428557-f751-4d98-abc8-7d1e12a09593","Type":"ContainerStarted","Data":"166b4430f2a0f0d8f03be5abb884ec25eba640549785bfa674beb1e6cc9eea18"} Jul 18 00:41:34.110523 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.110483 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:41:34.113500 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.113479 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 00:41:34.124047 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.124016 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8f576bc-9ad2-47cd-a879-43ac2e334a95-metrics-certs\") pod \"network-metrics-daemon-2zfkq\" (UID: \"a8f576bc-9ad2-47cd-a879-43ac2e334a95\") " pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:41:34.211288 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.211251 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:41:34.214078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.214047 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 00:41:34.225093 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.225058 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/55cbdaea-d9a9-4eca-b1c0-65de273699dd-original-pull-secret\") pod \"global-pull-secret-syncer-vhhcd\" (UID: \"55cbdaea-d9a9-4eca-b1c0-65de273699dd\") " pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:41:34.263520 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.263486 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vhhcd" Jul 18 00:41:34.279585 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.279553 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9tdp4\"" Jul 18 00:41:34.287517 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.287487 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zfkq" Jul 18 00:41:34.312682 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.312649 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx9j9\" (UniqueName: \"kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9\") pod \"network-check-target-hdxtn\" (UID: \"01540c57-73db-4206-bdd9-d83fbcbb8066\") " pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:41:34.315278 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.315254 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx9j9\" (UniqueName: \"kubernetes.io/projected/01540c57-73db-4206-bdd9-d83fbcbb8066-kube-api-access-cx9j9\") pod \"network-check-target-hdxtn\" (UID: \"01540c57-73db-4206-bdd9-d83fbcbb8066\") " pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:41:34.573254 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.573222 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2stqj\"" Jul 18 00:41:34.581497 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:34.581460 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:41:36.333371 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.333334 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:36.335756 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.335736 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7fea5857-2011-457a-91ad-c830f4686d0e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-67bw6\" (UID: \"7fea5857-2011-457a-91ad-c830f4686d0e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:36.434324 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.434288 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:36.434503 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.434333 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:36.434503 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.434378 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:36.434503 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.434408 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:36.434503 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.434450 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:36.434702 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.434530 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:36.436226 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.435147 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecfb0dac-7382-4175-804d-abd202994a71-service-ca-bundle\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:36.441026 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.437753 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecfb0dac-7382-4175-804d-abd202994a71-metrics-certs\") pod \"router-default-559bddd8f8-w24db\" (UID: \"ecfb0dac-7382-4175-804d-abd202994a71\") " pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:36.441026 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.437936 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/70547afb-1be6-4358-b0cd-5349d7dbf273-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kdk54\" (UID: \"70547afb-1be6-4358-b0cd-5349d7dbf273\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:36.441026 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.438068 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f12d587d-fdc5-4173-a8a3-a066a8f8b848-metrics-tls\") pod \"dns-default-2gm4c\" (UID: \"f12d587d-fdc5-4173-a8a3-a066a8f8b848\") " pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:36.441026 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.438139 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bdb58d5a-a4ac-435f-92ae-ff17de63bb0f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbdj\" (UID: \"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:36.441026 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.438526 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls\") pod \"image-registry-5dfcf5fff6-2h49l\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:36.536471 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.536440 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tz56m\"" Jul 18 00:41:36.544397 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.544376 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" Jul 18 00:41:36.583257 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.583232 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nfcpz\"" Jul 18 00:41:36.591162 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.591091 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:36.597801 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.597771 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rgftg\"" Jul 18 00:41:36.605883 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.605860 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" Jul 18 00:41:36.608341 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.608325 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hd2wz\"" Jul 18 00:41:36.616516 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.616498 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:36.620123 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.620105 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j7vqp\"" Jul 18 00:41:36.628501 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.628480 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:36.644434 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.644414 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8nvk6\"" Jul 18 00:41:36.652443 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:36.652417 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" Jul 18 00:41:37.141598 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.141472 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hdxtn"] Jul 18 00:41:37.386444 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:37.386414 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01540c57_73db_4206_bdd9_d83fbcbb8066.slice/crio-04888cfe6cee25f74945e2f536beba7ba5141c58d761436e77f232d9fcc9e5b3 WatchSource:0}: Error finding container 04888cfe6cee25f74945e2f536beba7ba5141c58d761436e77f232d9fcc9e5b3: Status 404 returned error can't find the container with id 04888cfe6cee25f74945e2f536beba7ba5141c58d761436e77f232d9fcc9e5b3 Jul 18 00:41:37.458088 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.457647 2585 scope.go:117] "RemoveContainer" containerID="848edffb52bdbd8dadbbbea877851ec2130b6803e1195d1b9bd8564a0193245f" Jul 18 00:41:37.590271 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.590198 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vhhcd"] Jul 18 00:41:37.611467 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.611172 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2zfkq"] Jul 18 00:41:37.611739 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:37.611583 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55cbdaea_d9a9_4eca_b1c0_65de273699dd.slice/crio-8df9a4733588f7be19c00263a764a9cc3578b08217c1e90973c065afc4b5755f WatchSource:0}: Error finding container 8df9a4733588f7be19c00263a764a9cc3578b08217c1e90973c065afc4b5755f: Status 404 returned error can't find the container with id 8df9a4733588f7be19c00263a764a9cc3578b08217c1e90973c065afc4b5755f Jul 18 00:41:37.626315 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:37.626221 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f576bc_9ad2_47cd_a879_43ac2e334a95.slice/crio-c68d3284ce7d385104678b0d51c15bff5da71f42ac9d03d6e67d681c7998ec9e WatchSource:0}: Error finding container c68d3284ce7d385104678b0d51c15bff5da71f42ac9d03d6e67d681c7998ec9e: Status 404 returned error can't find the container with id c68d3284ce7d385104678b0d51c15bff5da71f42ac9d03d6e67d681c7998ec9e Jul 18 00:41:37.681761 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:37.681725 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb58d5a_a4ac_435f_92ae_ff17de63bb0f.slice/crio-9c6649ba0f90d086d4cb6c2dd20b6c75027013aeeb3434659689e3aac8be6d73 WatchSource:0}: Error finding container 9c6649ba0f90d086d4cb6c2dd20b6c75027013aeeb3434659689e3aac8be6d73: Status 404 returned error can't find the container with id 9c6649ba0f90d086d4cb6c2dd20b6c75027013aeeb3434659689e3aac8be6d73 Jul 18 00:41:37.685009 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.684917 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj"] Jul 18 00:41:37.693478 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.693401 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6"] Jul 18 00:41:37.721065 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.720986 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dfcf5fff6-2h49l"] Jul 18 00:41:37.731161 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:37.731111 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9a89e63_02c2_495f_b2d5_72742bc08595.slice/crio-640ec7b8f9b24fed1d4b4b0fadad40a91633f0da2c005420437939423d1e98a2 WatchSource:0}: Error finding container 640ec7b8f9b24fed1d4b4b0fadad40a91633f0da2c005420437939423d1e98a2: Status 404 returned error can't find the container with id 640ec7b8f9b24fed1d4b4b0fadad40a91633f0da2c005420437939423d1e98a2 Jul 18 00:41:37.732027 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.731971 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-559bddd8f8-w24db"] Jul 18 00:41:37.737130 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:37.737056 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecfb0dac_7382_4175_804d_abd202994a71.slice/crio-9737c6619c6f23068b85013582c847cf104fb9836989d2e6ff162b7a339d437b WatchSource:0}: Error finding container 9737c6619c6f23068b85013582c847cf104fb9836989d2e6ff162b7a339d437b: Status 404 returned error can't find the container with id 9737c6619c6f23068b85013582c847cf104fb9836989d2e6ff162b7a339d437b Jul 18 00:41:37.751122 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.751091 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54"] Jul 18 00:41:37.754891 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.754869 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2gm4c"] Jul 18 00:41:37.757691 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:37.757647 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70547afb_1be6_4358_b0cd_5349d7dbf273.slice/crio-aab5728d81e835f731e85c547e149ccedd9d59e23aeef68d7ce2fe29ba53c3ab WatchSource:0}: Error finding container aab5728d81e835f731e85c547e149ccedd9d59e23aeef68d7ce2fe29ba53c3ab: Status 404 returned error can't find the container with id aab5728d81e835f731e85c547e149ccedd9d59e23aeef68d7ce2fe29ba53c3ab Jul 18 00:41:37.759268 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:37.759243 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12d587d_fdc5_4173_a8a3_a066a8f8b848.slice/crio-4cdfa30d530e69afa05f2bbb45050d57aab44b9f1ad8980a2416037efe0b382e WatchSource:0}: Error finding container 4cdfa30d530e69afa05f2bbb45050d57aab44b9f1ad8980a2416037efe0b382e: Status 404 returned error can't find the container with id 4cdfa30d530e69afa05f2bbb45050d57aab44b9f1ad8980a2416037efe0b382e Jul 18 00:41:37.915605 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.915558 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" event={"ID":"7fea5857-2011-457a-91ad-c830f4686d0e","Type":"ContainerStarted","Data":"14da4c4fb807470366b2c32ee17d2158c0dcfc66bc4d847ca5c38fedb8c15015"} Jul 18 00:41:37.916817 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.916786 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vhhcd" event={"ID":"55cbdaea-d9a9-4eca-b1c0-65de273699dd","Type":"ContainerStarted","Data":"8df9a4733588f7be19c00263a764a9cc3578b08217c1e90973c065afc4b5755f"} Jul 18 00:41:37.918134 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.918101 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" event={"ID":"70547afb-1be6-4358-b0cd-5349d7dbf273","Type":"ContainerStarted","Data":"aab5728d81e835f731e85c547e149ccedd9d59e23aeef68d7ce2fe29ba53c3ab"} Jul 18 00:41:37.919613 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.919591 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-559bddd8f8-w24db" event={"ID":"ecfb0dac-7382-4175-804d-abd202994a71","Type":"ContainerStarted","Data":"f9ab6fb198375bf03b91c99996ee2a8c7a081c129e0f95907a6085151bebe590"} Jul 18 00:41:37.919715 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.919632 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-559bddd8f8-w24db" event={"ID":"ecfb0dac-7382-4175-804d-abd202994a71","Type":"ContainerStarted","Data":"9737c6619c6f23068b85013582c847cf104fb9836989d2e6ff162b7a339d437b"} Jul 18 00:41:37.920801 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.920774 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2zfkq" event={"ID":"a8f576bc-9ad2-47cd-a879-43ac2e334a95","Type":"ContainerStarted","Data":"c68d3284ce7d385104678b0d51c15bff5da71f42ac9d03d6e67d681c7998ec9e"} Jul 18 00:41:37.922701 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.922683 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 00:41:37.922791 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.922752 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" event={"ID":"15ccd479-3004-4964-a298-0c510fc4bf9b","Type":"ContainerStarted","Data":"b2c09f17a6500615de4deb184516be6b09b4b98ef576d823bac39713b08a9d5b"} Jul 18 00:41:37.923024 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.923010 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:37.924253 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.924229 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2gm4c" event={"ID":"f12d587d-fdc5-4173-a8a3-a066a8f8b848","Type":"ContainerStarted","Data":"4cdfa30d530e69afa05f2bbb45050d57aab44b9f1ad8980a2416037efe0b382e"} Jul 18 00:41:37.925759 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.925740 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" event={"ID":"a9a89e63-02c2-495f-b2d5-72742bc08595","Type":"ContainerStarted","Data":"cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81"} Jul 18 00:41:37.925880 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.925775 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" event={"ID":"a9a89e63-02c2-495f-b2d5-72742bc08595","Type":"ContainerStarted","Data":"640ec7b8f9b24fed1d4b4b0fadad40a91633f0da2c005420437939423d1e98a2"} Jul 18 00:41:37.925880 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.925800 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:41:37.927284 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.927258 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f2qcd" event={"ID":"cc428557-f751-4d98-abc8-7d1e12a09593","Type":"ContainerStarted","Data":"f197064ae97cf2d72bb0c37c9b7e918d515d98b66e743ab95714c6a601f32252"} Jul 18 00:41:37.928446 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.928420 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" event={"ID":"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f","Type":"ContainerStarted","Data":"9c6649ba0f90d086d4cb6c2dd20b6c75027013aeeb3434659689e3aac8be6d73"} Jul 18 00:41:37.929856 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.929824 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hdxtn" event={"ID":"01540c57-73db-4206-bdd9-d83fbcbb8066","Type":"ContainerStarted","Data":"07d2b5320cec0a64f82afbb2a518a648206bff81d14e1144b2ae52196ca103f7"} Jul 18 00:41:37.930039 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.929868 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hdxtn" event={"ID":"01540c57-73db-4206-bdd9-d83fbcbb8066","Type":"ContainerStarted","Data":"04888cfe6cee25f74945e2f536beba7ba5141c58d761436e77f232d9fcc9e5b3"} Jul 18 00:41:37.930039 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.929986 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:41:37.937818 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.937272 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-559bddd8f8-w24db" podStartSLOduration=57.937255005 podStartE2EDuration="57.937255005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:37.936762604 +0000 UTC m=+68.958897390" watchObservedRunningTime="2026-07-18 00:41:37.937255005 +0000 UTC m=+68.959389793" Jul 18 00:41:37.953992 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.953934 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" podStartSLOduration=50.360320222 podStartE2EDuration="57.953914142s" totalImagesPullingTime="7.59359392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:37.953583237 +0000 UTC m=+68.975718025" watchObservedRunningTime="2026-07-18 00:41:37.953914142 +0000 UTC m=+68.976048927" Jul 18 00:41:37.974422 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.974363 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" podStartSLOduration=57.974341531 podStartE2EDuration="57.974341531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:37.974242963 +0000 UTC m=+68.996377752" watchObservedRunningTime="2026-07-18 00:41:37.974341531 +0000 UTC m=+68.996476316" Jul 18 00:41:37.988348 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:37.988297 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hdxtn" podStartSLOduration=67.988280026 podStartE2EDuration="1m7.988280026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:37.987755688 +0000 UTC m=+69.009890472" watchObservedRunningTime="2026-07-18 00:41:37.988280026 +0000 UTC m=+69.010414811" Jul 18 00:41:38.038904 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:38.038870 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-kgsmq" Jul 18 00:41:38.278329 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:38.278086 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:38.380599 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:38.380448 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:38.516031 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:38.515943 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:38.593526 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:38.592281 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:38.595410 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:38.595193 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:38.707634 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:38.707587 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:38.915635 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:38.915584 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:38.942546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:38.942506 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:38.944057 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:38.943865 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-559bddd8f8-w24db" Jul 18 00:41:38.979996 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:38.979905 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:39.034387 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:39.034270 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:39.135566 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:39.135533 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:40.120808 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.120617 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:40.315113 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.314885 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:40.431555 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.430778 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:40.481097 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.481063 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5dfcf5fff6-2h49l"] Jul 18 00:41:40.506589 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.506556 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl"] Jul 18 00:41:40.512558 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.512531 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.517691 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.517667 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 18 00:41:40.518198 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.518012 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 18 00:41:40.518302 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.518231 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 18 00:41:40.518494 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.518315 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 18 00:41:40.520537 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.520507 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 18 00:41:40.522403 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.522350 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 18 00:41:40.529655 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.529629 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 18 00:41:40.530606 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.530582 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl"] Jul 18 00:41:40.606080 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.606036 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67ccfcbdb9-w4t85"] Jul 18 00:41:40.609502 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.609479 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.616798 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.616702 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:40.622729 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.622691 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67ccfcbdb9-w4t85"] Jul 18 00:41:40.674260 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.674227 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.674260 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.674271 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-hub\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.674522 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.674349 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.674522 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.674396 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-ca\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.674522 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.674416 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/00325e30-94d6-420f-a3b4-f440839ac11d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.674522 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.674438 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cppgr\" (UniqueName: \"kubernetes.io/projected/00325e30-94d6-420f-a3b4-f440839ac11d-kube-api-access-cppgr\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.713144 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.713060 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:40.775347 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775308 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-ca\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.775540 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775365 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/00325e30-94d6-420f-a3b4-f440839ac11d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.775540 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775383 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cppgr\" (UniqueName: \"kubernetes.io/projected/00325e30-94d6-420f-a3b4-f440839ac11d-kube-api-access-cppgr\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.775540 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775406 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/022ce19a-fe97-4101-bda1-2bf053b10ca2-bound-sa-token\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.775540 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775472 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.775757 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775559 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/022ce19a-fe97-4101-bda1-2bf053b10ca2-ca-trust-extracted\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.775757 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775590 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhbbr\" (UniqueName: \"kubernetes.io/projected/022ce19a-fe97-4101-bda1-2bf053b10ca2-kube-api-access-vhbbr\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.775757 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775630 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-hub\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.775757 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775688 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/022ce19a-fe97-4101-bda1-2bf053b10ca2-trusted-ca\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.775757 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775727 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.776041 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775762 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/022ce19a-fe97-4101-bda1-2bf053b10ca2-registry-certificates\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.776041 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775792 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/022ce19a-fe97-4101-bda1-2bf053b10ca2-image-registry-private-configuration\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.776041 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775872 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/022ce19a-fe97-4101-bda1-2bf053b10ca2-installation-pull-secrets\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.776041 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.775918 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/022ce19a-fe97-4101-bda1-2bf053b10ca2-registry-tls\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.776248 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.776210 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/00325e30-94d6-420f-a3b4-f440839ac11d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.778875 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.778809 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-hub\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.779033 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.779006 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.779208 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.779186 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.779359 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.779340 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/00325e30-94d6-420f-a3b4-f440839ac11d-ca\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.794555 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.794534 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cppgr\" (UniqueName: \"kubernetes.io/projected/00325e30-94d6-420f-a3b4-f440839ac11d-kube-api-access-cppgr\") pod \"cluster-proxy-proxy-agent-856c548ddb-srlzl\" (UID: \"00325e30-94d6-420f-a3b4-f440839ac11d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.836554 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.836491 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:40.850372 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.850339 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" Jul 18 00:41:40.876472 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.876439 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/022ce19a-fe97-4101-bda1-2bf053b10ca2-registry-certificates\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.876674 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.876484 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/022ce19a-fe97-4101-bda1-2bf053b10ca2-image-registry-private-configuration\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.876674 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.876518 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/022ce19a-fe97-4101-bda1-2bf053b10ca2-installation-pull-secrets\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.876674 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.876573 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/022ce19a-fe97-4101-bda1-2bf053b10ca2-registry-tls\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.876674 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.876661 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/022ce19a-fe97-4101-bda1-2bf053b10ca2-bound-sa-token\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.876940 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.876784 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/022ce19a-fe97-4101-bda1-2bf053b10ca2-ca-trust-extracted\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.876940 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.876820 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhbbr\" (UniqueName: \"kubernetes.io/projected/022ce19a-fe97-4101-bda1-2bf053b10ca2-kube-api-access-vhbbr\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.876940 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.876895 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/022ce19a-fe97-4101-bda1-2bf053b10ca2-trusted-ca\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.877516 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.877494 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/022ce19a-fe97-4101-bda1-2bf053b10ca2-registry-certificates\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.877724 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.877705 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/022ce19a-fe97-4101-bda1-2bf053b10ca2-trusted-ca\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.877885 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.877824 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/022ce19a-fe97-4101-bda1-2bf053b10ca2-ca-trust-extracted\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.880705 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.880662 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/022ce19a-fe97-4101-bda1-2bf053b10ca2-image-registry-private-configuration\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.880845 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.880719 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/022ce19a-fe97-4101-bda1-2bf053b10ca2-installation-pull-secrets\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.881956 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.881913 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/022ce19a-fe97-4101-bda1-2bf053b10ca2-registry-tls\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.886361 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.886319 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhbbr\" (UniqueName: \"kubernetes.io/projected/022ce19a-fe97-4101-bda1-2bf053b10ca2-kube-api-access-vhbbr\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.887800 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.887773 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/022ce19a-fe97-4101-bda1-2bf053b10ca2-bound-sa-token\") pod \"image-registry-67ccfcbdb9-w4t85\" (UID: \"022ce19a-fe97-4101-bda1-2bf053b10ca2\") " pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.922567 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.922527 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:40.936246 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:40.936155 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:41.141099 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:41.140937 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:41.201204 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:41.201141 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:41.477290 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:41.477251 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:41.583189 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:41.583148 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:44.973070 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:44.973027 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl"] Jul 18 00:41:45.735973 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:45.735596 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00325e30_94d6_420f_a3b4_f440839ac11d.slice/crio-219d479d94cabd9a93b375e578a33723cd659cff1f596a129a4c623c53fda1ee WatchSource:0}: Error finding container 219d479d94cabd9a93b375e578a33723cd659cff1f596a129a4c623c53fda1ee: Status 404 returned error can't find the container with id 219d479d94cabd9a93b375e578a33723cd659cff1f596a129a4c623c53fda1ee Jul 18 00:41:45.836390 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:45.836350 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:45.866266 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:45.866232 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67ccfcbdb9-w4t85"] Jul 18 00:41:45.891125 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:45.891088 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:45.900581 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:45.900554 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod022ce19a_fe97_4101_bda1_2bf053b10ca2.slice/crio-035160b5421d7a23ea035d64d8d255bf8263eecc20e73b7972fad9d499e0ad66 WatchSource:0}: Error finding container 035160b5421d7a23ea035d64d8d255bf8263eecc20e73b7972fad9d499e0ad66: Status 404 returned error can't find the container with id 035160b5421d7a23ea035d64d8d255bf8263eecc20e73b7972fad9d499e0ad66 Jul 18 00:41:45.969065 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:45.969030 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" event={"ID":"00325e30-94d6-420f-a3b4-f440839ac11d","Type":"ContainerStarted","Data":"219d479d94cabd9a93b375e578a33723cd659cff1f596a129a4c623c53fda1ee"} Jul 18 00:41:45.970413 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:45.970381 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" event={"ID":"022ce19a-fe97-4101-bda1-2bf053b10ca2","Type":"ContainerStarted","Data":"035160b5421d7a23ea035d64d8d255bf8263eecc20e73b7972fad9d499e0ad66"} Jul 18 00:41:46.441628 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.441548 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:46.539096 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.538951 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:46.643673 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.643584 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:46.707112 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.707058 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:46.808221 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.808069 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:46.946785 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.946733 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:46.980082 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.980044 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f2qcd" event={"ID":"cc428557-f751-4d98-abc8-7d1e12a09593","Type":"ContainerStarted","Data":"04c87cee0ef5e86fc15cd7997fba5edcf2184fbf818ddd34de083ede59809d90"} Jul 18 00:41:46.981963 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.981902 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" event={"ID":"bdb58d5a-a4ac-435f-92ae-ff17de63bb0f","Type":"ContainerStarted","Data":"26d1a86cb20e4d9fcee14e2418ededdcb6637158a62e1cab6863a522a17d9c18"} Jul 18 00:41:46.986336 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.986302 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" event={"ID":"022ce19a-fe97-4101-bda1-2bf053b10ca2","Type":"ContainerStarted","Data":"9d14337be33d173d08e99655d1817c796dd75af0fbc9e783171bc393b0f52b5c"} Jul 18 00:41:46.986564 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.986545 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:41:46.989655 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.989157 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" event={"ID":"70547afb-1be6-4358-b0cd-5349d7dbf273","Type":"ContainerStarted","Data":"fa3bd15fe78cb40afae77ad2044ec011d6e0486e95a0f83aadd447b37ea34d45"} Jul 18 00:41:46.994141 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.994025 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2zfkq" event={"ID":"a8f576bc-9ad2-47cd-a879-43ac2e334a95","Type":"ContainerStarted","Data":"1c33a888f654cb896b9f73d3f951d23aa714ca29e3bd27276d1b2ae295e53627"} Jul 18 00:41:46.994141 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.994074 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2zfkq" event={"ID":"a8f576bc-9ad2-47cd-a879-43ac2e334a95","Type":"ContainerStarted","Data":"ba16d31db017f42bbd2c753050a1d47d37279beba4128f1928bd2eeceb80addd"} Jul 18 00:41:46.997745 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.997714 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2gm4c" event={"ID":"f12d587d-fdc5-4173-a8a3-a066a8f8b848","Type":"ContainerStarted","Data":"7897fd1de8f68a4aea0f3697f4c3e75165cf887e21e1c0bd7c24cbee40b1d882"} Jul 18 00:41:46.997865 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.997773 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2gm4c" event={"ID":"f12d587d-fdc5-4173-a8a3-a066a8f8b848","Type":"ContainerStarted","Data":"486a9a1d66e24aa69fbc6f141fa40598993ef2115402d3ff750cff3c94803678"} Jul 18 00:41:46.998333 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:46.998308 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:47.001747 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.001117 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbdj" podStartSLOduration=58.788883016 podStartE2EDuration="1m7.001097657s" totalImagesPullingTime="8.212214641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:46.998994929 +0000 UTC m=+78.021129715" watchObservedRunningTime="2026-07-18 00:41:47.001097657 +0000 UTC m=+78.023232459" Jul 18 00:41:47.001747 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.001605 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" event={"ID":"7fea5857-2011-457a-91ad-c830f4686d0e","Type":"ContainerStarted","Data":"c3712c15f1bf2d7c710ccfaf4adb94f66cea9864961eeaff5286ab7444716317"} Jul 18 00:41:47.001747 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.001659 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" event={"ID":"7fea5857-2011-457a-91ad-c830f4686d0e","Type":"ContainerStarted","Data":"7d352cc8cb6d73e95d029123606182c12ce3595e390d784d3628b91d584e9939"} Jul 18 00:41:47.003867 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.003796 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vhhcd" event={"ID":"55cbdaea-d9a9-4eca-b1c0-65de273699dd","Type":"ContainerStarted","Data":"a55c65ef36ea8866332414cb52d2d1f18e384578458a0b5381b19bd63459cf2d"} Jul 18 00:41:47.018305 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.018222 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kdk54" podStartSLOduration=53.940194458 podStartE2EDuration="1m1.018199519s" totalImagesPullingTime="7.078005061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:47.015720207 +0000 UTC m=+78.037855016" watchObservedRunningTime="2026-07-18 00:41:47.018199519 +0000 UTC m=+78.040334284" Jul 18 00:41:47.035533 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.034494 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2zfkq" podStartSLOduration=69.942574332 podStartE2EDuration="1m17.034475159s" totalImagesPullingTime="7.091900827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:47.034044144 +0000 UTC m=+78.056178929" watchObservedRunningTime="2026-07-18 00:41:47.034475159 +0000 UTC m=+78.056609944" Jul 18 00:41:47.063410 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.063064 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2gm4c" podStartSLOduration=34.925555429 podStartE2EDuration="43.063041912s" totalImagesPullingTime="8.137486483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:47.062855358 +0000 UTC m=+78.084990140" watchObservedRunningTime="2026-07-18 00:41:47.063041912 +0000 UTC m=+78.085176698" Jul 18 00:41:47.086057 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.084875 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" podStartSLOduration=7.084854996 podStartE2EDuration="7.084854996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:47.083721234 +0000 UTC m=+78.105856018" watchObservedRunningTime="2026-07-18 00:41:47.084854996 +0000 UTC m=+78.106989761" Jul 18 00:41:47.100098 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.099959 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-67bw6" podStartSLOduration=60.023714378 podStartE2EDuration="1m7.099937463s" totalImagesPullingTime="7.076223085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:47.099790017 +0000 UTC m=+78.121924805" watchObservedRunningTime="2026-07-18 00:41:47.099937463 +0000 UTC m=+78.122072247" Jul 18 00:41:47.128556 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.128514 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:47.397858 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.397795 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:47.498412 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:47.498360 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:48.393550 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:48.393493 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:48.786511 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:48.786472 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:48.840213 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:48.840169 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:50.662003 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:50.661947 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:50.763486 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:50.763448 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:51.076440 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:51.076344 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:52.028475 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:52.028436 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" event={"ID":"00325e30-94d6-420f-a3b4-f440839ac11d","Type":"ContainerStarted","Data":"f38ae5e1ed3334d093745aa9f9211d3dbd734e64c4a7917e1385597400e895b3"} Jul 18 00:41:52.028475 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:52.028481 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" event={"ID":"00325e30-94d6-420f-a3b4-f440839ac11d","Type":"ContainerStarted","Data":"24e1b732caa00e3d6255072825b7d6e33aa34422bc954086442db163e4baf6a0"} Jul 18 00:41:52.028999 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:52.028493 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" event={"ID":"00325e30-94d6-420f-a3b4-f440839ac11d","Type":"ContainerStarted","Data":"457948486e91a17338aab49e1fe1120bf1031952eeaa19cfc85e292d87c89ba1"} Jul 18 00:41:52.030602 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:52.030577 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f2qcd" event={"ID":"cc428557-f751-4d98-abc8-7d1e12a09593","Type":"ContainerStarted","Data":"4f1b0e5b122fe1f6b99b1b0bd2e56953e1b903aa38f0ee4cc38361f28b6cb5e8"} Jul 18 00:41:52.048512 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:52.048464 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vhhcd" podStartSLOduration=73.729885419 podStartE2EDuration="1m22.048449445s" totalImagesPullingTime="8.318564026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:47.116306899 +0000 UTC m=+78.138441683" watchObservedRunningTime="2026-07-18 00:41:52.048449445 +0000 UTC m=+83.070584272" Jul 18 00:41:52.048640 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:52.048533 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856c548ddb-srlzl" podStartSLOduration=6.364925129 podStartE2EDuration="12.048528933s" totalImagesPullingTime="5.683603804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:41:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:52.046783198 +0000 UTC m=+83.068918021" watchObservedRunningTime="2026-07-18 00:41:52.048528933 +0000 UTC m=+83.070663718" Jul 18 00:41:52.064440 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:52.064390 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-f2qcd" podStartSLOduration=17.993107305 podStartE2EDuration="37.064374105s" totalImagesPullingTime="19.0712668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 00:41:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:41:52.06396144 +0000 UTC m=+83.086096245" watchObservedRunningTime="2026-07-18 00:41:52.064374105 +0000 UTC m=+83.086508889" Jul 18 00:41:52.136265 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:52.136207 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:54.013026 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:54.012980 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:54.066706 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:54.066674 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:54.687693 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:54.687650 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:54.783145 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:54.783105 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:54.835107 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:54.835073 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:55.533357 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:55.533316 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:56.815372 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:56.815329 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:57.027530 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.027487 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:57.114375 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.114276 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rq2m9"] Jul 18 00:41:57.117575 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.117549 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.120957 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.120930 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 18 00:41:57.121123 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.120934 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 18 00:41:57.121468 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.121450 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 18 00:41:57.121522 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.121498 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 18 00:41:57.121573 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.121534 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wdls5\"" Jul 18 00:41:57.160158 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.160109 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:57.213407 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.213365 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rppz8\" (UniqueName: \"kubernetes.io/projected/922aee26-4b8e-4415-9e86-22eb40786d48-kube-api-access-rppz8\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.213585 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.213414 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/922aee26-4b8e-4415-9e86-22eb40786d48-sys\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.213585 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.213436 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-wtmp\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.213585 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.213496 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/922aee26-4b8e-4415-9e86-22eb40786d48-root\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.213585 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.213541 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-accelerators-collector-config\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.213714 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.213584 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-textfile\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.213714 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.213610 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-tls\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.213714 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.213634 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/922aee26-4b8e-4415-9e86-22eb40786d48-metrics-client-ca\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.213714 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.213651 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.245957 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.245921 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:57.314912 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.314878 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/922aee26-4b8e-4415-9e86-22eb40786d48-sys\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.314912 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.314910 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-wtmp\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315115 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.314935 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/922aee26-4b8e-4415-9e86-22eb40786d48-root\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315115 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.314985 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/922aee26-4b8e-4415-9e86-22eb40786d48-root\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315115 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.315040 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-accelerators-collector-config\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315115 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.315064 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-wtmp\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315115 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.315076 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-textfile\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315115 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.315064 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/922aee26-4b8e-4415-9e86-22eb40786d48-sys\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315351 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.315118 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-tls\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315351 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.315141 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/922aee26-4b8e-4415-9e86-22eb40786d48-metrics-client-ca\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315351 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.315156 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315351 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:57.315261 2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 18 00:41:57.315351 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.315276 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rppz8\" (UniqueName: \"kubernetes.io/projected/922aee26-4b8e-4415-9e86-22eb40786d48-kube-api-access-rppz8\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315351 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:57.315319 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-tls podName:922aee26-4b8e-4415-9e86-22eb40786d48 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:57.815299905 +0000 UTC m=+88.837434666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-tls") pod "node-exporter-rq2m9" (UID: "922aee26-4b8e-4415-9e86-22eb40786d48") : secret "node-exporter-tls" not found Jul 18 00:41:57.315606 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.315437 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-textfile\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315719 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.315702 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/922aee26-4b8e-4415-9e86-22eb40786d48-metrics-client-ca\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.315772 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.315720 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-accelerators-collector-config\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.317526 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.317505 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.323764 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.323739 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rppz8\" (UniqueName: \"kubernetes.io/projected/922aee26-4b8e-4415-9e86-22eb40786d48-kube-api-access-rppz8\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.402907 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.402875 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:57.501317 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.501272 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:57.641965 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.641926 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:57.763421 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.763322 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:57.819717 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:57.819685 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-tls\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:57.820106 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:57.819855 2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 18 00:41:57.820106 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:41:57.819918 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-tls podName:922aee26-4b8e-4415-9e86-22eb40786d48 nodeName:}" failed. No retries permitted until 2026-07-18 00:41:58.819903816 +0000 UTC m=+89.842038578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-tls") pod "node-exporter-rq2m9" (UID: "922aee26-4b8e-4415-9e86-22eb40786d48") : secret "node-exporter-tls" not found Jul 18 00:41:58.017483 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:58.017387 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2gm4c" Jul 18 00:41:58.073151 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:58.073100 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:58.145264 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:58.145221 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:58.270118 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:58.270013 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:58.370008 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:58.369974 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:58.828413 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:58.828377 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-tls\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:58.830669 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:58.830631 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/922aee26-4b8e-4415-9e86-22eb40786d48-node-exporter-tls\") pod \"node-exporter-rq2m9\" (UID: \"922aee26-4b8e-4415-9e86-22eb40786d48\") " pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:58.926974 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:58.926934 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rq2m9" Jul 18 00:41:58.937237 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:41:58.937202 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922aee26_4b8e_4415_9e86_22eb40786d48.slice/crio-e8867ecf99c74fb3e153b53d3d9d7991af003478d96a11a147c4d134fc64c5bb WatchSource:0}: Error finding container e8867ecf99c74fb3e153b53d3d9d7991af003478d96a11a147c4d134fc64c5bb: Status 404 returned error can't find the container with id e8867ecf99c74fb3e153b53d3d9d7991af003478d96a11a147c4d134fc64c5bb Jul 18 00:41:59.064101 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:59.064057 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rq2m9" event={"ID":"922aee26-4b8e-4415-9e86-22eb40786d48","Type":"ContainerStarted","Data":"e8867ecf99c74fb3e153b53d3d9d7991af003478d96a11a147c4d134fc64c5bb"} Jul 18 00:41:59.775405 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:59.775367 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:41:59.876878 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:41:59.876814 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:00.008205 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:00.008161 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:00.105396 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:00.105365 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:00.205626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:00.205588 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:00.324557 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:00.324461 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:00.487229 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:00.487196 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:42:01.072593 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:01.072554 2585 generic.go:356] "Generic (PLEG): container finished" podID="922aee26-4b8e-4415-9e86-22eb40786d48" containerID="7a537e74bc107740549db7ae8881191ee22de2a3ee5eb4a1423ad280ed2bfd3f" exitCode=0 Jul 18 00:42:01.073105 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:01.072607 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rq2m9" event={"ID":"922aee26-4b8e-4415-9e86-22eb40786d48","Type":"ContainerDied","Data":"7a537e74bc107740549db7ae8881191ee22de2a3ee5eb4a1423ad280ed2bfd3f"} Jul 18 00:42:01.152233 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:01.152207 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:01.254561 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:01.254530 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:01.935240 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:01.935198 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:01.995302 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:01.995264 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:02.078272 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:02.078232 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rq2m9" event={"ID":"922aee26-4b8e-4415-9e86-22eb40786d48","Type":"ContainerStarted","Data":"94af6c17ddd9828f1f77ecfb5de1f7662d13c2ee144634e96c42aed5ebe69106"} Jul 18 00:42:02.078757 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:02.078297 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rq2m9" event={"ID":"922aee26-4b8e-4415-9e86-22eb40786d48","Type":"ContainerStarted","Data":"e3b5744f0ce99dc456958a836a32a87566ce6e18fe80ca39418c2d7024f05a83"} Jul 18 00:42:02.098046 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:02.097995 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rq2m9" podStartSLOduration=3.921799659 podStartE2EDuration="5.097977912s" totalImagesPullingTime="1.109510525s" totalInitContainerRuntime="66.667728ms" isStatefulPod=true podCreationTimestamp="2026-07-18 00:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:42:02.096685358 +0000 UTC m=+93.118820154" watchObservedRunningTime="2026-07-18 00:42:02.097977912 +0000 UTC m=+93.120112697" Jul 18 00:42:02.204091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:02.204001 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:03.154076 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:03.154040 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:03.253946 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:03.253906 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:03.331010 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:03.330958 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:03.739690 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:03.739645 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:03.795538 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:03.795504 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:05.511879 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.511816 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" podUID="a9a89e63-02c2-495f-b2d5-72742bc08595" containerName="registry" containerID="cri-o://cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81" gracePeriod=30 Jul 18 00:42:05.763403 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.763338 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:42:05.784271 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.784243 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9a89e63-02c2-495f-b2d5-72742bc08595-ca-trust-extracted\") pod \"a9a89e63-02c2-495f-b2d5-72742bc08595\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " Jul 18 00:42:05.784447 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.784360 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wb5lr\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-kube-api-access-wb5lr\") pod \"a9a89e63-02c2-495f-b2d5-72742bc08595\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " Jul 18 00:42:05.784447 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.784409 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-trusted-ca\") pod \"a9a89e63-02c2-495f-b2d5-72742bc08595\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " Jul 18 00:42:05.784553 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.784449 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-certificates\") pod \"a9a89e63-02c2-495f-b2d5-72742bc08595\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " Jul 18 00:42:05.784553 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.784475 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-image-registry-private-configuration\") pod \"a9a89e63-02c2-495f-b2d5-72742bc08595\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " Jul 18 00:42:05.784644 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.784573 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls\") pod \"a9a89e63-02c2-495f-b2d5-72742bc08595\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " Jul 18 00:42:05.784644 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.784600 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-installation-pull-secrets\") pod \"a9a89e63-02c2-495f-b2d5-72742bc08595\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " Jul 18 00:42:05.784644 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.784631 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-bound-sa-token\") pod \"a9a89e63-02c2-495f-b2d5-72742bc08595\" (UID: \"a9a89e63-02c2-495f-b2d5-72742bc08595\") " Jul 18 00:42:05.785594 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.785537 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a9a89e63-02c2-495f-b2d5-72742bc08595" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 00:42:05.785594 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.785545 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a9a89e63-02c2-495f-b2d5-72742bc08595" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 00:42:05.788551 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.788495 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a9a89e63-02c2-495f-b2d5-72742bc08595" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 00:42:05.789210 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.788815 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a9a89e63-02c2-495f-b2d5-72742bc08595" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 00:42:05.789210 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.788822 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a9a89e63-02c2-495f-b2d5-72742bc08595" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 00:42:05.789210 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.788877 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a9a89e63-02c2-495f-b2d5-72742bc08595" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 00:42:05.789477 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.789270 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-kube-api-access-wb5lr" (OuterVolumeSpecName: "kube-api-access-wb5lr") pod "a9a89e63-02c2-495f-b2d5-72742bc08595" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595"). InnerVolumeSpecName "kube-api-access-wb5lr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 00:42:05.797399 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.797371 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9a89e63-02c2-495f-b2d5-72742bc08595-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a9a89e63-02c2-495f-b2d5-72742bc08595" (UID: "a9a89e63-02c2-495f-b2d5-72742bc08595"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 00:42:05.819093 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.819056 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:05.886314 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.886274 2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-certificates\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:42:05.886314 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.886309 2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-image-registry-private-configuration\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:42:05.886314 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.886322 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-registry-tls\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:42:05.886551 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.886333 2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9a89e63-02c2-495f-b2d5-72742bc08595-installation-pull-secrets\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:42:05.886551 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.886341 2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-bound-sa-token\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:42:05.886551 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.886350 2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9a89e63-02c2-495f-b2d5-72742bc08595-ca-trust-extracted\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:42:05.886551 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.886359 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb5lr\" (UniqueName: \"kubernetes.io/projected/a9a89e63-02c2-495f-b2d5-72742bc08595-kube-api-access-wb5lr\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:42:05.886551 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:05.886367 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9a89e63-02c2-495f-b2d5-72742bc08595-trusted-ca\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:42:06.093354 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:06.093260 2585 generic.go:356] "Generic (PLEG): container finished" podID="a9a89e63-02c2-495f-b2d5-72742bc08595" containerID="cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81" exitCode=0 Jul 18 00:42:06.093354 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:06.093303 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" Jul 18 00:42:06.093354 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:06.093327 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" event={"ID":"a9a89e63-02c2-495f-b2d5-72742bc08595","Type":"ContainerDied","Data":"cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81"} Jul 18 00:42:06.093632 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:06.093377 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dfcf5fff6-2h49l" event={"ID":"a9a89e63-02c2-495f-b2d5-72742bc08595","Type":"ContainerDied","Data":"640ec7b8f9b24fed1d4b4b0fadad40a91633f0da2c005420437939423d1e98a2"} Jul 18 00:42:06.093632 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:06.093403 2585 scope.go:117] "RemoveContainer" containerID="cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81" Jul 18 00:42:06.102556 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:06.102535 2585 scope.go:117] "RemoveContainer" containerID="cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81" Jul 18 00:42:06.102816 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:42:06.102795 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81\": container with ID starting with cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81 not found: ID does not exist" containerID="cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81" Jul 18 00:42:06.102915 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:06.102847 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81"} err="failed to get container status \"cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81\": rpc error: code = NotFound desc = could not find container \"cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81\": container with ID starting with cb61db35ffb0054713c0e7b4946a3ce4d3ae08c55fe8929281d334b245bc3f81 not found: ID does not exist" Jul 18 00:42:06.114219 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:06.114193 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5dfcf5fff6-2h49l"] Jul 18 00:42:06.118028 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:06.118004 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5dfcf5fff6-2h49l"] Jul 18 00:42:07.457119 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:07.457079 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9a89e63-02c2-495f-b2d5-72742bc08595" path="/var/lib/kubelet/pods/a9a89e63-02c2-495f-b2d5-72742bc08595/volumes" Jul 18 00:42:08.015551 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:08.015522 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67ccfcbdb9-w4t85" Jul 18 00:42:08.241528 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:08.241488 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:08.345082 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:08.344992 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:08.944143 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:08.944097 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hdxtn" Jul 18 00:42:09.010960 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:09.010922 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:09.111085 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:09.111046 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:09.434607 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:09.434557 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:09.490175 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:09.490134 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:14.831902 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:14.831867 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:14.990234 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:14.990200 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:18.503310 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:18.503252 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:18.560679 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:18.560647 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:24.015655 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:24.015616 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:24.117524 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:24.117491 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:30.185747 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:30.185660 2585 generic.go:356] "Generic (PLEG): container finished" podID="2f6b75f9-43a8-4d7c-9cd0-7a350a753397" containerID="81a4956c394d6823bc7dc2d9fe239a9686ebe9d1d068fb616ee8c19d1224e03e" exitCode=0 Jul 18 00:42:30.186217 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:30.185754 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" event={"ID":"2f6b75f9-43a8-4d7c-9cd0-7a350a753397","Type":"ContainerDied","Data":"81a4956c394d6823bc7dc2d9fe239a9686ebe9d1d068fb616ee8c19d1224e03e"} Jul 18 00:42:30.186557 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:30.186537 2585 scope.go:117] "RemoveContainer" containerID="81a4956c394d6823bc7dc2d9fe239a9686ebe9d1d068fb616ee8c19d1224e03e" Jul 18 00:42:30.518607 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:30.518567 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:30.619100 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:30.619062 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:30.674070 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:30.674041 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:31.191408 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:31.191370 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-hdjjz" event={"ID":"2f6b75f9-43a8-4d7c-9cd0-7a350a753397","Type":"ContainerStarted","Data":"e99c769c8a2c549dfda360f8d2da71f4edc86d6fc13898f300323cf82637201f"} Jul 18 00:42:32.248498 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:32.248454 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:35.206768 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:35.206733 2585 generic.go:356] "Generic (PLEG): container finished" podID="602eb3d3-135b-46da-8f9a-0061de2cc17b" containerID="932c452f132cddcca7f986e6cc494dacd6d001dfed511c7510ba6debf0f39abf" exitCode=0 Jul 18 00:42:35.207265 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:35.206803 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-z987w" event={"ID":"602eb3d3-135b-46da-8f9a-0061de2cc17b","Type":"ContainerDied","Data":"932c452f132cddcca7f986e6cc494dacd6d001dfed511c7510ba6debf0f39abf"} Jul 18 00:42:35.207331 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:35.207304 2585 scope.go:117] "RemoveContainer" containerID="932c452f132cddcca7f986e6cc494dacd6d001dfed511c7510ba6debf0f39abf" Jul 18 00:42:36.211887 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:36.211852 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-z987w" event={"ID":"602eb3d3-135b-46da-8f9a-0061de2cc17b","Type":"ContainerStarted","Data":"217e5f2aca96ca9a23478ce31e3d35b7e4916c5a18d62e294111e48089404bef"} Jul 18 00:42:39.015963 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:39.015932 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:39.075447 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:39.075412 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:42:50.263784 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:50.263745 2585 generic.go:356] "Generic (PLEG): container finished" podID="aa89d4ee-2ac6-4234-8641-a851e295524d" containerID="3b089f7ba51f336dbdd987245694f5ee823aeee9a20019bf26d76b3e01c4ec70" exitCode=0 Jul 18 00:42:50.264299 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:50.263816 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" event={"ID":"aa89d4ee-2ac6-4234-8641-a851e295524d","Type":"ContainerDied","Data":"3b089f7ba51f336dbdd987245694f5ee823aeee9a20019bf26d76b3e01c4ec70"} Jul 18 00:42:50.264299 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:50.264184 2585 scope.go:117] "RemoveContainer" containerID="3b089f7ba51f336dbdd987245694f5ee823aeee9a20019bf26d76b3e01c4ec70" Jul 18 00:42:51.268898 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:51.268860 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8j4fl" event={"ID":"aa89d4ee-2ac6-4234-8641-a851e295524d","Type":"ContainerStarted","Data":"c07ca0c0442568db9f82c10dd3af513b12a6c9653ddc5b0872ce8ca1ac71845b"} Jul 18 00:42:55.385503 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:42:55.385466 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:43:16.276198 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:43:16.276148 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:43:16.477142 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:43:16.477107 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:43:25.966547 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:43:25.966442 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:43:26.066327 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:43:26.066294 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:43:26.126388 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:43:26.126358 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:43:26.647033 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:43:26.646998 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:43:36.565218 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:43:36.565182 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:43:37.271070 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:43:37.271038 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:43:37.371708 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:43:37.371678 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:43:37.473193 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:43:37.473159 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:06.113958 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:06.113904 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:06.166424 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:06.166384 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:11.173857 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:11.173799 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:31.562760 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:31.562707 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:31.618085 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:31.618043 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:38.500010 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:38.499952 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:38.547541 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:38.547494 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:43.510296 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:43.510256 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:43.561153 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:43.561115 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:49.526988 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:49.526956 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:49.577425 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:49.577390 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:49.646202 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:49.646167 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:54.024109 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:54.024021 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:54.125807 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:54.125766 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:54.402511 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:54.402478 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:44:54.459090 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:44:54.459060 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:22.161928 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:22.161893 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:22.260314 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:22.260274 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:29.333260 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:29.333227 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 00:45:29.334662 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:29.334639 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 00:45:29.337109 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:29.337090 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 00:45:29.338514 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:29.338493 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 00:45:29.344624 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:29.344603 2585 kubelet.go:1648] "Image garbage collection succeeded" Jul 18 00:45:48.750869 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:48.750823 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:50.297647 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:50.297603 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:52.358656 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:52.358621 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:52.418306 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:52.418273 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:54.016405 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:54.016373 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:54.128909 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:54.128870 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:56.516820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:56.516786 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:56.703491 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:56.703452 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:59.157815 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:59.157769 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:45:59.217741 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:45:59.217706 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:01.752875 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:01.752839 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:01.859280 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:01.859246 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:03.673973 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:03.673939 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:04.971160 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:04.971117 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:05.067276 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:05.067239 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:10.394088 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:10.394052 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:10.452506 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:10.452463 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:12.454078 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:12.454044 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:28.371622 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:28.371517 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:28.469757 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:28.469704 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:28.973199 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:28.973170 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:29.173458 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.173423 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:29.275522 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.275403 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:29.365403 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.365366 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf"] Jul 18 00:46:29.365681 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.365667 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9a89e63-02c2-495f-b2d5-72742bc08595" containerName="registry" Jul 18 00:46:29.365727 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.365684 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9a89e63-02c2-495f-b2d5-72742bc08595" containerName="registry" Jul 18 00:46:29.365761 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.365730 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9a89e63-02c2-495f-b2d5-72742bc08595" containerName="registry" Jul 18 00:46:29.367689 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.367671 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.375606 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.375581 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 18 00:46:29.376015 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.375615 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 18 00:46:29.376015 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.375767 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-46br2\"" Jul 18 00:46:29.376015 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.375774 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 18 00:46:29.383336 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.383320 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 18 00:46:29.397758 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.397733 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf"] Jul 18 00:46:29.400395 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.400368 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8014a90-8142-4c1f-8396-917d8ab27b09-tmp\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.400483 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.400414 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a8014a90-8142-4c1f-8396-917d8ab27b09-manifests\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.400524 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.400488 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mggrx\" (UniqueName: \"kubernetes.io/projected/a8014a90-8142-4c1f-8396-917d8ab27b09-kube-api-access-mggrx\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.400612 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.400583 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8014a90-8142-4c1f-8396-917d8ab27b09-webhook-cert\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.400720 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.400632 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a8014a90-8142-4c1f-8396-917d8ab27b09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.501322 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.501291 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8014a90-8142-4c1f-8396-917d8ab27b09-webhook-cert\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.501530 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.501327 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a8014a90-8142-4c1f-8396-917d8ab27b09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.501530 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.501371 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8014a90-8142-4c1f-8396-917d8ab27b09-tmp\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.501530 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.501391 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a8014a90-8142-4c1f-8396-917d8ab27b09-manifests\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.501530 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.501419 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mggrx\" (UniqueName: \"kubernetes.io/projected/a8014a90-8142-4c1f-8396-917d8ab27b09-kube-api-access-mggrx\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.501923 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.501889 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a8014a90-8142-4c1f-8396-917d8ab27b09-manifests\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.503731 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.503703 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8014a90-8142-4c1f-8396-917d8ab27b09-tmp\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.503930 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.503911 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8014a90-8142-4c1f-8396-917d8ab27b09-webhook-cert\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.504003 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.503967 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a8014a90-8142-4c1f-8396-917d8ab27b09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.516433 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.516406 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mggrx\" (UniqueName: \"kubernetes.io/projected/a8014a90-8142-4c1f-8396-917d8ab27b09-kube-api-access-mggrx\") pod \"opendatahub-operator-controller-manager-85c6dd979f-p62qf\" (UID: \"a8014a90-8142-4c1f-8396-917d8ab27b09\") " pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.540626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.540562 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:29.644746 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.644708 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:29.677381 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.677349 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:29.741476 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.741426 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:29.809155 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.809061 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:29.825768 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.825745 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf"] Jul 18 00:46:29.827804 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:46:29.827778 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8014a90_8142_4c1f_8396_917d8ab27b09.slice/crio-1b532128ecb68a5df1bc41f8d999e609dadfa6a29c13fede5c849c78d8eb3da4 WatchSource:0}: Error finding container 1b532128ecb68a5df1bc41f8d999e609dadfa6a29c13fede5c849c78d8eb3da4: Status 404 returned error can't find the container with id 1b532128ecb68a5df1bc41f8d999e609dadfa6a29c13fede5c849c78d8eb3da4 Jul 18 00:46:29.829632 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.829611 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 00:46:29.873976 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.873948 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:29.971013 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:29.970974 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:30.073083 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:30.073001 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" event={"ID":"a8014a90-8142-4c1f-8396-917d8ab27b09","Type":"ContainerStarted","Data":"1b532128ecb68a5df1bc41f8d999e609dadfa6a29c13fede5c849c78d8eb3da4"} Jul 18 00:46:33.089143 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:33.089059 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" event={"ID":"a8014a90-8142-4c1f-8396-917d8ab27b09","Type":"ContainerStarted","Data":"e3c2dd05be0ad332e7df492ef46770dccb12743f64c5001846aca2ae6a937dc8"} Jul 18 00:46:34.094374 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:34.094337 2585 generic.go:356] "Generic (PLEG): container finished" podID="a8014a90-8142-4c1f-8396-917d8ab27b09" containerID="e3c2dd05be0ad332e7df492ef46770dccb12743f64c5001846aca2ae6a937dc8" exitCode=0 Jul 18 00:46:34.094768 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:34.094404 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" event={"ID":"a8014a90-8142-4c1f-8396-917d8ab27b09","Type":"ContainerDied","Data":"e3c2dd05be0ad332e7df492ef46770dccb12743f64c5001846aca2ae6a937dc8"} Jul 18 00:46:35.100123 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:35.100077 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" event={"ID":"a8014a90-8142-4c1f-8396-917d8ab27b09","Type":"ContainerStarted","Data":"5ad522ef6f79c82119e11ac3b68b4ad59780ef54b7051b9ec6922e4e432a7040"} Jul 18 00:46:35.100533 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:35.100191 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:35.122393 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:35.122333 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" podStartSLOduration=1.730404314 podStartE2EDuration="6.122318233s" totalImagesPullingTime="3.326753802s" totalInitContainerRuntime="1.065160117s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:46:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:46:35.120473904 +0000 UTC m=+366.142608689" watchObservedRunningTime="2026-07-18 00:46:35.122318233 +0000 UTC m=+366.144453016" Jul 18 00:46:38.638122 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:38.638059 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:38.753961 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:38.753928 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:40.895022 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:40.894981 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:41.004298 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:41.004260 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:46.106334 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:46.106306 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85c6dd979f-p62qf" Jul 18 00:46:48.583883 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:48.583814 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:48.701422 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:48.701386 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:49.395598 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.395559 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-29kpq"] Jul 18 00:46:49.405277 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.405250 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" Jul 18 00:46:49.409094 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.409071 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9xz86\"" Jul 18 00:46:49.409679 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.409664 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 18 00:46:49.415120 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.415092 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-29kpq"] Jul 18 00:46:49.453439 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.453402 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:49.553423 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.553378 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:49.577465 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.577423 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5kvd\" (UniqueName: \"kubernetes.io/projected/dfc0ce47-acdd-4caf-849f-4e7557ade323-kube-api-access-f5kvd\") pod \"odh-model-controller-7998c67dc4-29kpq\" (UID: \"dfc0ce47-acdd-4caf-849f-4e7557ade323\") " pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" Jul 18 00:46:49.577636 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.577480 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc0ce47-acdd-4caf-849f-4e7557ade323-cert\") pod \"odh-model-controller-7998c67dc4-29kpq\" (UID: \"dfc0ce47-acdd-4caf-849f-4e7557ade323\") " pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" Jul 18 00:46:49.672782 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.672679 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:49.678160 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.678125 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5kvd\" (UniqueName: \"kubernetes.io/projected/dfc0ce47-acdd-4caf-849f-4e7557ade323-kube-api-access-f5kvd\") pod \"odh-model-controller-7998c67dc4-29kpq\" (UID: \"dfc0ce47-acdd-4caf-849f-4e7557ade323\") " pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" Jul 18 00:46:49.678311 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.678175 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc0ce47-acdd-4caf-849f-4e7557ade323-cert\") pod \"odh-model-controller-7998c67dc4-29kpq\" (UID: \"dfc0ce47-acdd-4caf-849f-4e7557ade323\") " pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" Jul 18 00:46:49.680678 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.680656 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc0ce47-acdd-4caf-849f-4e7557ade323-cert\") pod \"odh-model-controller-7998c67dc4-29kpq\" (UID: \"dfc0ce47-acdd-4caf-849f-4e7557ade323\") " pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" Jul 18 00:46:49.710116 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.710082 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5kvd\" (UniqueName: \"kubernetes.io/projected/dfc0ce47-acdd-4caf-849f-4e7557ade323-kube-api-access-f5kvd\") pod \"odh-model-controller-7998c67dc4-29kpq\" (UID: \"dfc0ce47-acdd-4caf-849f-4e7557ade323\") " pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" Jul 18 00:46:49.715285 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.715258 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" Jul 18 00:46:49.794202 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.794146 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:49.829364 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.829325 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m"] Jul 18 00:46:49.832650 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.832624 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" Jul 18 00:46:49.835755 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.835726 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 18 00:46:49.835942 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.835922 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 18 00:46:49.836028 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.835936 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8fvc4\"" Jul 18 00:46:49.842095 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.842031 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m"] Jul 18 00:46:49.852342 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.852314 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-29kpq"] Jul 18 00:46:49.857447 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:46:49.857415 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfc0ce47_acdd_4caf_849f_4e7557ade323.slice/crio-3b75d079ce0bc3cf92ddfcfb9cb85c4f2b4c0f1bbbd0e6d593536e6ace75216c WatchSource:0}: Error finding container 3b75d079ce0bc3cf92ddfcfb9cb85c4f2b4c0f1bbbd0e6d593536e6ace75216c: Status 404 returned error can't find the container with id 3b75d079ce0bc3cf92ddfcfb9cb85c4f2b4c0f1bbbd0e6d593536e6ace75216c Jul 18 00:46:49.981139 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.981050 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ffaa630-baec-47ee-9418-fc56b7b206ba-tmp\") pod \"kube-auth-proxy-5f655d47ff-bmr2m\" (UID: \"2ffaa630-baec-47ee-9418-fc56b7b206ba\") " pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" Jul 18 00:46:49.981139 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.981089 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ffaa630-baec-47ee-9418-fc56b7b206ba-tls-certs\") pod \"kube-auth-proxy-5f655d47ff-bmr2m\" (UID: \"2ffaa630-baec-47ee-9418-fc56b7b206ba\") " pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" Jul 18 00:46:49.981139 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:49.981128 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kc2m\" (UniqueName: \"kubernetes.io/projected/2ffaa630-baec-47ee-9418-fc56b7b206ba-kube-api-access-2kc2m\") pod \"kube-auth-proxy-5f655d47ff-bmr2m\" (UID: \"2ffaa630-baec-47ee-9418-fc56b7b206ba\") " pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" Jul 18 00:46:50.071843 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.071772 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:50.081558 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.081535 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kc2m\" (UniqueName: \"kubernetes.io/projected/2ffaa630-baec-47ee-9418-fc56b7b206ba-kube-api-access-2kc2m\") pod \"kube-auth-proxy-5f655d47ff-bmr2m\" (UID: \"2ffaa630-baec-47ee-9418-fc56b7b206ba\") " pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" Jul 18 00:46:50.081688 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.081606 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ffaa630-baec-47ee-9418-fc56b7b206ba-tmp\") pod \"kube-auth-proxy-5f655d47ff-bmr2m\" (UID: \"2ffaa630-baec-47ee-9418-fc56b7b206ba\") " pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" Jul 18 00:46:50.081688 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.081628 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ffaa630-baec-47ee-9418-fc56b7b206ba-tls-certs\") pod \"kube-auth-proxy-5f655d47ff-bmr2m\" (UID: \"2ffaa630-baec-47ee-9418-fc56b7b206ba\") " pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" Jul 18 00:46:50.083968 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.083932 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ffaa630-baec-47ee-9418-fc56b7b206ba-tmp\") pod \"kube-auth-proxy-5f655d47ff-bmr2m\" (UID: \"2ffaa630-baec-47ee-9418-fc56b7b206ba\") " pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" Jul 18 00:46:50.084088 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.084071 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ffaa630-baec-47ee-9418-fc56b7b206ba-tls-certs\") pod \"kube-auth-proxy-5f655d47ff-bmr2m\" (UID: \"2ffaa630-baec-47ee-9418-fc56b7b206ba\") " pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" Jul 18 00:46:50.090397 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.090369 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kc2m\" (UniqueName: \"kubernetes.io/projected/2ffaa630-baec-47ee-9418-fc56b7b206ba-kube-api-access-2kc2m\") pod \"kube-auth-proxy-5f655d47ff-bmr2m\" (UID: \"2ffaa630-baec-47ee-9418-fc56b7b206ba\") " pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" Jul 18 00:46:50.149334 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.149297 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" Jul 18 00:46:50.173413 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.173376 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" event={"ID":"dfc0ce47-acdd-4caf-849f-4e7557ade323","Type":"ContainerStarted","Data":"3b75d079ce0bc3cf92ddfcfb9cb85c4f2b4c0f1bbbd0e6d593536e6ace75216c"} Jul 18 00:46:50.196274 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.196241 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:50.294004 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.293968 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m"] Jul 18 00:46:50.296926 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:46:50.296895 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ffaa630_baec_47ee_9418_fc56b7b206ba.slice/crio-874031a71ae0bf52083a6d5968dc3cdcbfb2d971c75c24338da4f31406a2d4c5 WatchSource:0}: Error finding container 874031a71ae0bf52083a6d5968dc3cdcbfb2d971c75c24338da4f31406a2d4c5: Status 404 returned error can't find the container with id 874031a71ae0bf52083a6d5968dc3cdcbfb2d971c75c24338da4f31406a2d4c5 Jul 18 00:46:50.505054 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:50.504958 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:51.034720 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:51.034531 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:51.145107 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:51.145048 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:51.182447 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:51.182397 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" event={"ID":"2ffaa630-baec-47ee-9418-fc56b7b206ba","Type":"ContainerStarted","Data":"874031a71ae0bf52083a6d5968dc3cdcbfb2d971c75c24338da4f31406a2d4c5"} Jul 18 00:46:52.136881 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:52.136843 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:54.199566 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:54.199528 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" event={"ID":"2ffaa630-baec-47ee-9418-fc56b7b206ba","Type":"ContainerStarted","Data":"9b21a104a5ea342f94f531f9d136512ee5c1535fa9f25b8e46c221ec7eeedac3"} Jul 18 00:46:54.201024 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:54.200998 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" event={"ID":"dfc0ce47-acdd-4caf-849f-4e7557ade323","Type":"ContainerStarted","Data":"0b1f0458d23c9756a0b84398143210a6fb1f16a03595e1d9e62e955993710ac5"} Jul 18 00:46:54.201161 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:54.201137 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" Jul 18 00:46:54.225588 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:54.225532 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f655d47ff-bmr2m" podStartSLOduration=1.950576637 podStartE2EDuration="5.225517508s" totalImagesPullingTime="3.274940871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:46:54.224094421 +0000 UTC m=+385.246229211" watchObservedRunningTime="2026-07-18 00:46:54.225517508 +0000 UTC m=+385.247652285" Jul 18 00:46:54.243886 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:54.243783 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" podStartSLOduration=1.5252372539999999 podStartE2EDuration="5.243765606s" totalImagesPullingTime="3.718528352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:46:54.242743595 +0000 UTC m=+385.264878380" watchObservedRunningTime="2026-07-18 00:46:54.243765606 +0000 UTC m=+385.265900389" Jul 18 00:46:55.524874 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:55.524817 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:55.623965 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:55.623919 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:56.246572 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:56.246543 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:56.451549 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:56.451505 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:56.567345 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:56.567246 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:46:57.986814 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:57.986777 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh"] Jul 18 00:46:57.990298 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:57.990275 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:57.993004 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:57.992978 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh"] Jul 18 00:46:57.993696 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:57.993679 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 18 00:46:57.994946 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:57.994926 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5nglk\"" Jul 18 00:46:57.995113 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:57.994928 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 18 00:46:57.995113 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:57.994939 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 18 00:46:57.995285 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:57.994943 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 18 00:46:57.995285 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:57.994988 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 18 00:46:58.158256 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.158219 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c103c660-6c37-4855-8aad-ad0557eb272a-manager-config\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.158256 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.158264 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c103c660-6c37-4855-8aad-ad0557eb272a-cert\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.158498 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.158310 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm7d4\" (UniqueName: \"kubernetes.io/projected/c103c660-6c37-4855-8aad-ad0557eb272a-kube-api-access-wm7d4\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.158498 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.158454 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c103c660-6c37-4855-8aad-ad0557eb272a-metrics-cert\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.259045 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.258941 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c103c660-6c37-4855-8aad-ad0557eb272a-manager-config\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.259045 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.258997 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c103c660-6c37-4855-8aad-ad0557eb272a-cert\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.259292 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.259115 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm7d4\" (UniqueName: \"kubernetes.io/projected/c103c660-6c37-4855-8aad-ad0557eb272a-kube-api-access-wm7d4\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.259292 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.259220 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c103c660-6c37-4855-8aad-ad0557eb272a-metrics-cert\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.259612 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.259585 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c103c660-6c37-4855-8aad-ad0557eb272a-manager-config\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.261605 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.261576 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c103c660-6c37-4855-8aad-ad0557eb272a-metrics-cert\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.261708 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.261657 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c103c660-6c37-4855-8aad-ad0557eb272a-cert\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.284895 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.284854 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm7d4\" (UniqueName: \"kubernetes.io/projected/c103c660-6c37-4855-8aad-ad0557eb272a-kube-api-access-wm7d4\") pod \"lws-controller-manager-5b88d5d556-mksdh\" (UID: \"c103c660-6c37-4855-8aad-ad0557eb272a\") " pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.301543 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.301510 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:46:58.461972 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:58.461792 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh"] Jul 18 00:46:58.465058 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:46:58.465024 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc103c660_6c37_4855_8aad_ad0557eb272a.slice/crio-899b892faf0c31f6efd585ee38a6ff01a83c04e19a7de3047475844ae5c6b093 WatchSource:0}: Error finding container 899b892faf0c31f6efd585ee38a6ff01a83c04e19a7de3047475844ae5c6b093: Status 404 returned error can't find the container with id 899b892faf0c31f6efd585ee38a6ff01a83c04e19a7de3047475844ae5c6b093 Jul 18 00:46:59.224768 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:46:59.224722 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" event={"ID":"c103c660-6c37-4855-8aad-ad0557eb272a","Type":"ContainerStarted","Data":"899b892faf0c31f6efd585ee38a6ff01a83c04e19a7de3047475844ae5c6b093"} Jul 18 00:47:01.083142 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:01.083108 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:01.234998 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:01.234964 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:01.235806 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:01.235777 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" event={"ID":"c103c660-6c37-4855-8aad-ad0557eb272a","Type":"ContainerStarted","Data":"7585e4a986166a8d7591be0cf5ca43526f1efa269a88d7852b8d16808a3a2828"} Jul 18 00:47:01.235981 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:01.235884 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:47:01.296778 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:01.296720 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" podStartSLOduration=2.234374112 podStartE2EDuration="4.296705589s" totalImagesPullingTime="2.062331477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:47:01.295898971 +0000 UTC m=+392.318033753" watchObservedRunningTime="2026-07-18 00:47:01.296705589 +0000 UTC m=+392.318840373" Jul 18 00:47:01.480248 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:01.480215 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:02.301382 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:02.301347 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:02.500703 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:02.500668 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:02.666443 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:02.666412 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:02.815786 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:02.815742 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:05.208104 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:05.208073 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-29kpq" Jul 18 00:47:12.243231 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:12.243198 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b88d5d556-mksdh" Jul 18 00:47:49.922724 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:49.922680 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:50.026220 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:50.026185 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:50.084990 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:50.084945 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:50.828292 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:50.828260 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:51.039297 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:51.039264 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:47:54.253513 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:47:54.253424 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:08.011820 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:08.011770 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:08.064769 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:08.064732 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:22.038501 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:22.038464 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:26.264206 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:26.264150 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:26.822529 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:26.822489 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:26.920133 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:26.920098 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:27.020593 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:27.020562 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:27.526584 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:27.526549 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:27.592951 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:27.592903 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:28.848104 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:28.848068 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:30.490779 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:30.490749 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:31.923236 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:31.923180 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:33.444241 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:33.444203 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:33.726352 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:33.726258 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:34.116328 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:34.116233 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:36.308900 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:36.308855 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:36.415478 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:36.415435 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:37.687504 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:37.687471 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:38.544388 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:38.544355 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:38.644305 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:38.644268 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:40.459321 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:40.459284 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:40.555030 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:40.554992 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:41.741761 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:41.741731 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:41.887963 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:41.887925 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:45.664757 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:45.664712 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:45.746419 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:45.746377 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:54.202245 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:54.202205 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:54.297912 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:54.297867 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:56.111111 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:56.111073 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:56.174348 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:56.174316 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:56.789088 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:56.789054 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:56.892059 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:56.892028 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:57.965384 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:57.965346 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:58.022772 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:58.022741 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:59.783380 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:59.783341 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:48:59.882875 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:48:59.882817 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:03.598748 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:03.598708 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:03.663982 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:03.663944 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:12.182363 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:12.182325 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:12.279853 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:12.279792 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:33.088644 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:33.088560 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:33.222158 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:33.222120 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:48.689226 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:48.689179 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:48.795041 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:48.795005 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:48.895924 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:48.895881 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:49.502787 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.502754 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-9q7j8"] Jul 18 00:49:49.505320 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.505302 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9q7j8" Jul 18 00:49:49.508057 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.508032 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 00:49:49.508057 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.508032 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 00:49:49.508982 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.508966 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r57sl\"" Jul 18 00:49:49.513246 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.513220 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx6hl\" (UniqueName: \"kubernetes.io/projected/8d0e79e0-df0b-450d-b75a-95c6e6216877-kube-api-access-wx6hl\") pod \"authorino-58cd955675-9q7j8\" (UID: \"8d0e79e0-df0b-450d-b75a-95c6e6216877\") " pod="kuadrant-system/authorino-58cd955675-9q7j8" Jul 18 00:49:49.514799 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.514776 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-9q7j8"] Jul 18 00:49:49.544539 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.544498 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:49.614029 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.613979 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx6hl\" (UniqueName: \"kubernetes.io/projected/8d0e79e0-df0b-450d-b75a-95c6e6216877-kube-api-access-wx6hl\") pod \"authorino-58cd955675-9q7j8\" (UID: \"8d0e79e0-df0b-450d-b75a-95c6e6216877\") " pod="kuadrant-system/authorino-58cd955675-9q7j8" Jul 18 00:49:49.622388 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.622359 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx6hl\" (UniqueName: \"kubernetes.io/projected/8d0e79e0-df0b-450d-b75a-95c6e6216877-kube-api-access-wx6hl\") pod \"authorino-58cd955675-9q7j8\" (UID: \"8d0e79e0-df0b-450d-b75a-95c6e6216877\") " pod="kuadrant-system/authorino-58cd955675-9q7j8" Jul 18 00:49:49.642298 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.642265 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:49:49.719450 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.719423 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r62p"] Jul 18 00:49:49.721619 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.721596 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8r62p" Jul 18 00:49:49.731796 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.731765 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r62p"] Jul 18 00:49:49.815457 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.815368 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9q7j8" Jul 18 00:49:49.815615 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.815583 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5lvc\" (UniqueName: \"kubernetes.io/projected/d0e3d7dd-0983-4b21-8f52-437e834b5e1f-kube-api-access-w5lvc\") pod \"authorino-75d556bd77-8r62p\" (UID: \"d0e3d7dd-0983-4b21-8f52-437e834b5e1f\") " pod="kuadrant-system/authorino-75d556bd77-8r62p" Jul 18 00:49:49.917044 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.917006 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5lvc\" (UniqueName: \"kubernetes.io/projected/d0e3d7dd-0983-4b21-8f52-437e834b5e1f-kube-api-access-w5lvc\") pod \"authorino-75d556bd77-8r62p\" (UID: \"d0e3d7dd-0983-4b21-8f52-437e834b5e1f\") " pod="kuadrant-system/authorino-75d556bd77-8r62p" Jul 18 00:49:49.924786 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.924755 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5lvc\" (UniqueName: \"kubernetes.io/projected/d0e3d7dd-0983-4b21-8f52-437e834b5e1f-kube-api-access-w5lvc\") pod \"authorino-75d556bd77-8r62p\" (UID: \"d0e3d7dd-0983-4b21-8f52-437e834b5e1f\") " pod="kuadrant-system/authorino-75d556bd77-8r62p" Jul 18 00:49:49.943781 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:49.943752 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-9q7j8"] Jul 18 00:49:49.947707 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:49:49.947676 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d0e79e0_df0b_450d_b75a_95c6e6216877.slice/crio-cf313b508eca793d99330a2f52bdb24fbcd2e42808f249620ce8df6eddf4ade3 WatchSource:0}: Error finding container cf313b508eca793d99330a2f52bdb24fbcd2e42808f249620ce8df6eddf4ade3: Status 404 returned error can't find the container with id cf313b508eca793d99330a2f52bdb24fbcd2e42808f249620ce8df6eddf4ade3 Jul 18 00:49:50.031940 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:50.031898 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8r62p" Jul 18 00:49:50.162132 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:50.161969 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r62p"] Jul 18 00:49:50.164711 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:49:50.164682 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0e3d7dd_0983_4b21_8f52_437e834b5e1f.slice/crio-b3bfb87cb3d67fc9c3aaf751ddc67a87e7ae1badeffdd60a42592a1152641e3d WatchSource:0}: Error finding container b3bfb87cb3d67fc9c3aaf751ddc67a87e7ae1badeffdd60a42592a1152641e3d: Status 404 returned error can't find the container with id b3bfb87cb3d67fc9c3aaf751ddc67a87e7ae1badeffdd60a42592a1152641e3d Jul 18 00:49:50.948893 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:50.948792 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8r62p" event={"ID":"d0e3d7dd-0983-4b21-8f52-437e834b5e1f","Type":"ContainerStarted","Data":"b3bfb87cb3d67fc9c3aaf751ddc67a87e7ae1badeffdd60a42592a1152641e3d"} Jul 18 00:49:50.952486 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:50.952451 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9q7j8" event={"ID":"8d0e79e0-df0b-450d-b75a-95c6e6216877","Type":"ContainerStarted","Data":"cf313b508eca793d99330a2f52bdb24fbcd2e42808f249620ce8df6eddf4ade3"} Jul 18 00:49:53.968216 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:53.968174 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8r62p" event={"ID":"d0e3d7dd-0983-4b21-8f52-437e834b5e1f","Type":"ContainerStarted","Data":"9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6"} Jul 18 00:49:53.969669 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:53.969643 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9q7j8" event={"ID":"8d0e79e0-df0b-450d-b75a-95c6e6216877","Type":"ContainerStarted","Data":"3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936"} Jul 18 00:49:53.994598 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:53.994546 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-8r62p" podStartSLOduration=1.971128977 podStartE2EDuration="4.994530974s" totalImagesPullingTime="3.023401997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:49:53.994265298 +0000 UTC m=+565.016400082" watchObservedRunningTime="2026-07-18 00:49:53.994530974 +0000 UTC m=+565.016665755" Jul 18 00:49:54.029533 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:54.029474 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-9q7j8" podStartSLOduration=1.794144942 podStartE2EDuration="5.029459116s" totalImagesPullingTime="3.235314174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:49:54.028003374 +0000 UTC m=+565.050138159" watchObservedRunningTime="2026-07-18 00:49:54.029459116 +0000 UTC m=+565.051593899" Jul 18 00:49:54.059680 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:54.059646 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-9q7j8"] Jul 18 00:49:55.979246 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:55.979203 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-9q7j8" podUID="8d0e79e0-df0b-450d-b75a-95c6e6216877" containerName="authorino" containerID="cri-o://3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936" gracePeriod=30 Jul 18 00:49:56.235705 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:56.235641 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9q7j8" Jul 18 00:49:56.272960 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:56.272915 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx6hl\" (UniqueName: \"kubernetes.io/projected/8d0e79e0-df0b-450d-b75a-95c6e6216877-kube-api-access-wx6hl\") pod \"8d0e79e0-df0b-450d-b75a-95c6e6216877\" (UID: \"8d0e79e0-df0b-450d-b75a-95c6e6216877\") " Jul 18 00:49:56.275124 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:56.275090 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d0e79e0-df0b-450d-b75a-95c6e6216877-kube-api-access-wx6hl" (OuterVolumeSpecName: "kube-api-access-wx6hl") pod "8d0e79e0-df0b-450d-b75a-95c6e6216877" (UID: "8d0e79e0-df0b-450d-b75a-95c6e6216877"). InnerVolumeSpecName "kube-api-access-wx6hl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 00:49:56.374228 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:56.374176 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx6hl\" (UniqueName: \"kubernetes.io/projected/8d0e79e0-df0b-450d-b75a-95c6e6216877-kube-api-access-wx6hl\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:49:56.984503 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:56.984463 2585 generic.go:356] "Generic (PLEG): container finished" podID="8d0e79e0-df0b-450d-b75a-95c6e6216877" containerID="3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936" exitCode=0 Jul 18 00:49:56.984933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:56.984510 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-9q7j8" Jul 18 00:49:56.984933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:56.984527 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9q7j8" event={"ID":"8d0e79e0-df0b-450d-b75a-95c6e6216877","Type":"ContainerDied","Data":"3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936"} Jul 18 00:49:56.984933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:56.984567 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-9q7j8" event={"ID":"8d0e79e0-df0b-450d-b75a-95c6e6216877","Type":"ContainerDied","Data":"cf313b508eca793d99330a2f52bdb24fbcd2e42808f249620ce8df6eddf4ade3"} Jul 18 00:49:56.984933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:56.984591 2585 scope.go:117] "RemoveContainer" containerID="3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936" Jul 18 00:49:56.993686 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:56.993664 2585 scope.go:117] "RemoveContainer" containerID="3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936" Jul 18 00:49:56.993959 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:49:56.993943 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936\": container with ID starting with 3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936 not found: ID does not exist" containerID="3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936" Jul 18 00:49:56.994027 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:56.993969 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936"} err="failed to get container status \"3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936\": rpc error: code = NotFound desc = could not find container \"3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936\": container with ID starting with 3c69af5089575e46559f0d2c77d1a8c8711c7aeddfc37422903ee5cb38107936 not found: ID does not exist" Jul 18 00:49:57.004325 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:57.004297 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-9q7j8"] Jul 18 00:49:57.006428 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:57.006409 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-9q7j8"] Jul 18 00:49:57.456478 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:49:57.456444 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d0e79e0-df0b-450d-b75a-95c6e6216877" path="/var/lib/kubelet/pods/8d0e79e0-df0b-450d-b75a-95c6e6216877/volumes" Jul 18 00:50:06.132947 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:06.132909 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:06.233173 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:06.233134 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:06.577049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:06.577016 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:18.253122 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.253084 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:18.353490 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.353458 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:18.479996 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.479959 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6f2rk"] Jul 18 00:50:18.480619 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.480597 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d0e79e0-df0b-450d-b75a-95c6e6216877" containerName="authorino" Jul 18 00:50:18.480728 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.480673 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d0e79e0-df0b-450d-b75a-95c6e6216877" containerName="authorino" Jul 18 00:50:18.480806 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.480782 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d0e79e0-df0b-450d-b75a-95c6e6216877" containerName="authorino" Jul 18 00:50:18.485132 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.485106 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6f2rk" Jul 18 00:50:18.492227 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.491966 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6f2rk"] Jul 18 00:50:18.571943 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.571815 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwtcv\" (UniqueName: \"kubernetes.io/projected/65d234a5-180b-4ed5-b005-08c0e7cf47de-kube-api-access-jwtcv\") pod \"authorino-5d4b67c446-6f2rk\" (UID: \"65d234a5-180b-4ed5-b005-08c0e7cf47de\") " pod="kuadrant-system/authorino-5d4b67c446-6f2rk" Jul 18 00:50:18.673291 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.673251 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwtcv\" (UniqueName: \"kubernetes.io/projected/65d234a5-180b-4ed5-b005-08c0e7cf47de-kube-api-access-jwtcv\") pod \"authorino-5d4b67c446-6f2rk\" (UID: \"65d234a5-180b-4ed5-b005-08c0e7cf47de\") " pod="kuadrant-system/authorino-5d4b67c446-6f2rk" Jul 18 00:50:18.683352 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.683316 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwtcv\" (UniqueName: \"kubernetes.io/projected/65d234a5-180b-4ed5-b005-08c0e7cf47de-kube-api-access-jwtcv\") pod \"authorino-5d4b67c446-6f2rk\" (UID: \"65d234a5-180b-4ed5-b005-08c0e7cf47de\") " pod="kuadrant-system/authorino-5d4b67c446-6f2rk" Jul 18 00:50:18.711387 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.711354 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6f2rk"] Jul 18 00:50:18.711643 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.711629 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6f2rk" Jul 18 00:50:18.740292 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.740261 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5577659957-rgz7v"] Jul 18 00:50:18.743809 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.743790 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5577659957-rgz7v" Jul 18 00:50:18.749785 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.749757 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5577659957-rgz7v"] Jul 18 00:50:18.766805 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.766025 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5577659957-rgz7v"] Jul 18 00:50:18.766805 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:50:18.766330 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-kqg6n], unattached volumes=[], failed to process volumes=[kube-api-access-kqg6n]: context canceled" pod="kuadrant-system/authorino-5577659957-rgz7v" podUID="8fce5fe8-5cc7-4f08-bc19-5159974113ba" Jul 18 00:50:18.788053 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.788017 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-ff84948f7-kldkx"] Jul 18 00:50:18.791250 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.791225 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff84948f7-kldkx" Jul 18 00:50:18.793418 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.793396 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 18 00:50:18.799805 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.799779 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-ff84948f7-kldkx"] Jul 18 00:50:18.853537 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.853510 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6f2rk"] Jul 18 00:50:18.856018 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:50:18.855987 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65d234a5_180b_4ed5_b005_08c0e7cf47de.slice/crio-ffac56e948a909a88a862d7648da103acb3a384ad75fffe9e2cd54dbcb2ecbc9 WatchSource:0}: Error finding container ffac56e948a909a88a862d7648da103acb3a384ad75fffe9e2cd54dbcb2ecbc9: Status 404 returned error can't find the container with id ffac56e948a909a88a862d7648da103acb3a384ad75fffe9e2cd54dbcb2ecbc9 Jul 18 00:50:18.876138 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.876102 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a158b93-a53e-42d9-b654-e29f00136e35-tls-cert\") pod \"authorino-ff84948f7-kldkx\" (UID: \"9a158b93-a53e-42d9-b654-e29f00136e35\") " pod="kuadrant-system/authorino-ff84948f7-kldkx" Jul 18 00:50:18.876269 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.876176 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqg6n\" (UniqueName: \"kubernetes.io/projected/8fce5fe8-5cc7-4f08-bc19-5159974113ba-kube-api-access-kqg6n\") pod \"authorino-5577659957-rgz7v\" (UID: \"8fce5fe8-5cc7-4f08-bc19-5159974113ba\") " pod="kuadrant-system/authorino-5577659957-rgz7v" Jul 18 00:50:18.876269 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.876227 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2rnx\" (UniqueName: \"kubernetes.io/projected/9a158b93-a53e-42d9-b654-e29f00136e35-kube-api-access-t2rnx\") pod \"authorino-ff84948f7-kldkx\" (UID: \"9a158b93-a53e-42d9-b654-e29f00136e35\") " pod="kuadrant-system/authorino-ff84948f7-kldkx" Jul 18 00:50:18.976929 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.976898 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqg6n\" (UniqueName: \"kubernetes.io/projected/8fce5fe8-5cc7-4f08-bc19-5159974113ba-kube-api-access-kqg6n\") pod \"authorino-5577659957-rgz7v\" (UID: \"8fce5fe8-5cc7-4f08-bc19-5159974113ba\") " pod="kuadrant-system/authorino-5577659957-rgz7v" Jul 18 00:50:18.977098 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.976942 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2rnx\" (UniqueName: \"kubernetes.io/projected/9a158b93-a53e-42d9-b654-e29f00136e35-kube-api-access-t2rnx\") pod \"authorino-ff84948f7-kldkx\" (UID: \"9a158b93-a53e-42d9-b654-e29f00136e35\") " pod="kuadrant-system/authorino-ff84948f7-kldkx" Jul 18 00:50:18.977098 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.976994 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a158b93-a53e-42d9-b654-e29f00136e35-tls-cert\") pod \"authorino-ff84948f7-kldkx\" (UID: \"9a158b93-a53e-42d9-b654-e29f00136e35\") " pod="kuadrant-system/authorino-ff84948f7-kldkx" Jul 18 00:50:18.979402 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.979383 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a158b93-a53e-42d9-b654-e29f00136e35-tls-cert\") pod \"authorino-ff84948f7-kldkx\" (UID: \"9a158b93-a53e-42d9-b654-e29f00136e35\") " pod="kuadrant-system/authorino-ff84948f7-kldkx" Jul 18 00:50:18.984461 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.984431 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2rnx\" (UniqueName: \"kubernetes.io/projected/9a158b93-a53e-42d9-b654-e29f00136e35-kube-api-access-t2rnx\") pod \"authorino-ff84948f7-kldkx\" (UID: \"9a158b93-a53e-42d9-b654-e29f00136e35\") " pod="kuadrant-system/authorino-ff84948f7-kldkx" Jul 18 00:50:18.984610 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:18.984594 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqg6n\" (UniqueName: \"kubernetes.io/projected/8fce5fe8-5cc7-4f08-bc19-5159974113ba-kube-api-access-kqg6n\") pod \"authorino-5577659957-rgz7v\" (UID: \"8fce5fe8-5cc7-4f08-bc19-5159974113ba\") " pod="kuadrant-system/authorino-5577659957-rgz7v" Jul 18 00:50:19.085999 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:19.085972 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5577659957-rgz7v" Jul 18 00:50:19.086166 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:19.085971 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6f2rk" event={"ID":"65d234a5-180b-4ed5-b005-08c0e7cf47de","Type":"ContainerStarted","Data":"ffac56e948a909a88a862d7648da103acb3a384ad75fffe9e2cd54dbcb2ecbc9"} Jul 18 00:50:19.093772 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:19.093750 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5577659957-rgz7v" Jul 18 00:50:19.101324 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:19.101245 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff84948f7-kldkx" Jul 18 00:50:19.178755 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:19.178721 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqg6n\" (UniqueName: \"kubernetes.io/projected/8fce5fe8-5cc7-4f08-bc19-5159974113ba-kube-api-access-kqg6n\") pod \"8fce5fe8-5cc7-4f08-bc19-5159974113ba\" (UID: \"8fce5fe8-5cc7-4f08-bc19-5159974113ba\") " Jul 18 00:50:19.180888 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:19.180846 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fce5fe8-5cc7-4f08-bc19-5159974113ba-kube-api-access-kqg6n" (OuterVolumeSpecName: "kube-api-access-kqg6n") pod "8fce5fe8-5cc7-4f08-bc19-5159974113ba" (UID: "8fce5fe8-5cc7-4f08-bc19-5159974113ba"). InnerVolumeSpecName "kube-api-access-kqg6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 00:50:19.234172 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:19.234007 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-ff84948f7-kldkx"] Jul 18 00:50:19.238974 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:50:19.238940 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a158b93_a53e_42d9_b654_e29f00136e35.slice/crio-074a58749ef09637b35c1ba9318e53c1a9f94b8e0acaad2f7e13cb6f374ba3cf WatchSource:0}: Error finding container 074a58749ef09637b35c1ba9318e53c1a9f94b8e0acaad2f7e13cb6f374ba3cf: Status 404 returned error can't find the container with id 074a58749ef09637b35c1ba9318e53c1a9f94b8e0acaad2f7e13cb6f374ba3cf Jul 18 00:50:19.283875 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:19.280639 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqg6n\" (UniqueName: \"kubernetes.io/projected/8fce5fe8-5cc7-4f08-bc19-5159974113ba-kube-api-access-kqg6n\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:50:20.091822 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.091780 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6f2rk" event={"ID":"65d234a5-180b-4ed5-b005-08c0e7cf47de","Type":"ContainerStarted","Data":"8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954"} Jul 18 00:50:20.092137 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.091897 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-6f2rk" podUID="65d234a5-180b-4ed5-b005-08c0e7cf47de" containerName="authorino" containerID="cri-o://8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954" gracePeriod=30 Jul 18 00:50:20.093496 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.093463 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff84948f7-kldkx" event={"ID":"9a158b93-a53e-42d9-b654-e29f00136e35","Type":"ContainerStarted","Data":"c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a"} Jul 18 00:50:20.093634 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.093511 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff84948f7-kldkx" event={"ID":"9a158b93-a53e-42d9-b654-e29f00136e35","Type":"ContainerStarted","Data":"074a58749ef09637b35c1ba9318e53c1a9f94b8e0acaad2f7e13cb6f374ba3cf"} Jul 18 00:50:20.093634 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.093557 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5577659957-rgz7v" Jul 18 00:50:20.106786 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.106743 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-6f2rk" podStartSLOduration=1.751894823 podStartE2EDuration="2.106729125s" totalImagesPullingTime="354.834302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:50:20.10472471 +0000 UTC m=+591.126859494" watchObservedRunningTime="2026-07-18 00:50:20.106729125 +0000 UTC m=+591.128863966" Jul 18 00:50:20.137656 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.137622 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5577659957-rgz7v"] Jul 18 00:50:20.141737 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.141712 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5577659957-rgz7v"] Jul 18 00:50:20.155295 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.155247 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-ff84948f7-kldkx" podStartSLOduration=1.833949401 podStartE2EDuration="2.155230405s" totalImagesPullingTime="321.281004ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:50:20.153869693 +0000 UTC m=+591.176004478" watchObservedRunningTime="2026-07-18 00:50:20.155230405 +0000 UTC m=+591.177365239" Jul 18 00:50:20.179532 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.179496 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r62p"] Jul 18 00:50:20.179708 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.179686 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-8r62p" podUID="d0e3d7dd-0983-4b21-8f52-437e834b5e1f" containerName="authorino" containerID="cri-o://9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6" gracePeriod=30 Jul 18 00:50:20.438845 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.438809 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6f2rk" Jul 18 00:50:20.443993 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.443969 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8r62p" Jul 18 00:50:20.591350 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.591318 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwtcv\" (UniqueName: \"kubernetes.io/projected/65d234a5-180b-4ed5-b005-08c0e7cf47de-kube-api-access-jwtcv\") pod \"65d234a5-180b-4ed5-b005-08c0e7cf47de\" (UID: \"65d234a5-180b-4ed5-b005-08c0e7cf47de\") " Jul 18 00:50:20.591520 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.591501 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5lvc\" (UniqueName: \"kubernetes.io/projected/d0e3d7dd-0983-4b21-8f52-437e834b5e1f-kube-api-access-w5lvc\") pod \"d0e3d7dd-0983-4b21-8f52-437e834b5e1f\" (UID: \"d0e3d7dd-0983-4b21-8f52-437e834b5e1f\") " Jul 18 00:50:20.593407 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.593372 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65d234a5-180b-4ed5-b005-08c0e7cf47de-kube-api-access-jwtcv" (OuterVolumeSpecName: "kube-api-access-jwtcv") pod "65d234a5-180b-4ed5-b005-08c0e7cf47de" (UID: "65d234a5-180b-4ed5-b005-08c0e7cf47de"). InnerVolumeSpecName "kube-api-access-jwtcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 00:50:20.593520 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.593409 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e3d7dd-0983-4b21-8f52-437e834b5e1f-kube-api-access-w5lvc" (OuterVolumeSpecName: "kube-api-access-w5lvc") pod "d0e3d7dd-0983-4b21-8f52-437e834b5e1f" (UID: "d0e3d7dd-0983-4b21-8f52-437e834b5e1f"). InnerVolumeSpecName "kube-api-access-w5lvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 00:50:20.692091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.692063 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5lvc\" (UniqueName: \"kubernetes.io/projected/d0e3d7dd-0983-4b21-8f52-437e834b5e1f-kube-api-access-w5lvc\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:50:20.692091 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:20.692090 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwtcv\" (UniqueName: \"kubernetes.io/projected/65d234a5-180b-4ed5-b005-08c0e7cf47de-kube-api-access-jwtcv\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:50:21.099658 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.099564 2585 generic.go:356] "Generic (PLEG): container finished" podID="d0e3d7dd-0983-4b21-8f52-437e834b5e1f" containerID="9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6" exitCode=0 Jul 18 00:50:21.099658 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.099606 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8r62p" Jul 18 00:50:21.099921 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.099649 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8r62p" event={"ID":"d0e3d7dd-0983-4b21-8f52-437e834b5e1f","Type":"ContainerDied","Data":"9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6"} Jul 18 00:50:21.099921 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.099697 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8r62p" event={"ID":"d0e3d7dd-0983-4b21-8f52-437e834b5e1f","Type":"ContainerDied","Data":"b3bfb87cb3d67fc9c3aaf751ddc67a87e7ae1badeffdd60a42592a1152641e3d"} Jul 18 00:50:21.099921 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.099719 2585 scope.go:117] "RemoveContainer" containerID="9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6" Jul 18 00:50:21.101087 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.100926 2585 generic.go:356] "Generic (PLEG): container finished" podID="65d234a5-180b-4ed5-b005-08c0e7cf47de" containerID="8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954" exitCode=0 Jul 18 00:50:21.101087 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.100989 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6f2rk" Jul 18 00:50:21.101087 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.101019 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6f2rk" event={"ID":"65d234a5-180b-4ed5-b005-08c0e7cf47de","Type":"ContainerDied","Data":"8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954"} Jul 18 00:50:21.101087 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.101055 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6f2rk" event={"ID":"65d234a5-180b-4ed5-b005-08c0e7cf47de","Type":"ContainerDied","Data":"ffac56e948a909a88a862d7648da103acb3a384ad75fffe9e2cd54dbcb2ecbc9"} Jul 18 00:50:21.109143 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.109099 2585 scope.go:117] "RemoveContainer" containerID="9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6" Jul 18 00:50:21.109423 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:50:21.109375 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6\": container with ID starting with 9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6 not found: ID does not exist" containerID="9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6" Jul 18 00:50:21.109423 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.109405 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6"} err="failed to get container status \"9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6\": rpc error: code = NotFound desc = could not find container \"9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6\": container with ID starting with 9ee804f141139f5d7bc863313df9b564ad2cf8776b654906ebdf05206c215ca6 not found: ID does not exist" Jul 18 00:50:21.109541 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.109429 2585 scope.go:117] "RemoveContainer" containerID="8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954" Jul 18 00:50:21.116924 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.116908 2585 scope.go:117] "RemoveContainer" containerID="8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954" Jul 18 00:50:21.117152 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:50:21.117135 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954\": container with ID starting with 8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954 not found: ID does not exist" containerID="8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954" Jul 18 00:50:21.117203 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.117163 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954"} err="failed to get container status \"8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954\": rpc error: code = NotFound desc = could not find container \"8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954\": container with ID starting with 8c963adf922dc7b8ec5dd8108d74b5e5d6999266fec7eee79cf62a9c2f419954 not found: ID does not exist" Jul 18 00:50:21.122555 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.122532 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6f2rk"] Jul 18 00:50:21.126694 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.126674 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6f2rk"] Jul 18 00:50:21.136126 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.136107 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r62p"] Jul 18 00:50:21.137803 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.137786 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8r62p"] Jul 18 00:50:21.457008 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.456978 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65d234a5-180b-4ed5-b005-08c0e7cf47de" path="/var/lib/kubelet/pods/65d234a5-180b-4ed5-b005-08c0e7cf47de/volumes" Jul 18 00:50:21.457393 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.457293 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fce5fe8-5cc7-4f08-bc19-5159974113ba" path="/var/lib/kubelet/pods/8fce5fe8-5cc7-4f08-bc19-5159974113ba/volumes" Jul 18 00:50:21.457480 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:21.457468 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e3d7dd-0983-4b21-8f52-437e834b5e1f" path="/var/lib/kubelet/pods/d0e3d7dd-0983-4b21-8f52-437e834b5e1f/volumes" Jul 18 00:50:22.617797 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:22.617756 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:22.718475 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:22.718442 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:24.685506 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:24.685473 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:24.767648 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:24.767614 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:24.850722 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:24.850684 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:27.955435 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:27.955402 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:28.062474 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:28.062438 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:29.359794 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:29.359768 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 00:50:29.360238 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:29.360100 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 00:50:29.364149 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:29.364124 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 00:50:29.364346 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:29.364330 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 00:50:54.006767 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:54.006671 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:59.799537 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:59.799495 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:59.889821 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:59.889776 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:50:59.990105 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:50:59.990065 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:00.381644 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:00.381607 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:00.588340 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:00.588300 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:00.885317 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:00.885274 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:00.983663 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:00.983627 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:10.967785 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:10.967750 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:11.063977 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:11.063944 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:21.971266 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:21.971226 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:34.129168 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.129130 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng"] Jul 18 00:51:34.129775 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.129447 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e3d7dd-0983-4b21-8f52-437e834b5e1f" containerName="authorino" Jul 18 00:51:34.129775 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.129462 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e3d7dd-0983-4b21-8f52-437e834b5e1f" containerName="authorino" Jul 18 00:51:34.129775 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.129484 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65d234a5-180b-4ed5-b005-08c0e7cf47de" containerName="authorino" Jul 18 00:51:34.129775 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.129490 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="65d234a5-180b-4ed5-b005-08c0e7cf47de" containerName="authorino" Jul 18 00:51:34.129775 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.129541 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e3d7dd-0983-4b21-8f52-437e834b5e1f" containerName="authorino" Jul 18 00:51:34.129775 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.129553 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="65d234a5-180b-4ed5-b005-08c0e7cf47de" containerName="authorino" Jul 18 00:51:34.132559 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.132534 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.134960 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.134933 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 18 00:51:34.135099 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.135035 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 18 00:51:34.135921 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.135889 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dl528\"" Jul 18 00:51:34.136010 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.135933 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 18 00:51:34.142950 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.142925 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng"] Jul 18 00:51:34.159249 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.159208 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:34.214498 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.214465 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf3016e2-a114-4579-b443-cb5429f0ce59-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.214671 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.214525 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.214671 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.214582 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.214671 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.214637 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.214814 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.214673 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.214814 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.214697 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jmpr\" (UniqueName: \"kubernetes.io/projected/cf3016e2-a114-4579-b443-cb5429f0ce59-kube-api-access-6jmpr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.315669 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.315637 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf3016e2-a114-4579-b443-cb5429f0ce59-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.315895 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.315707 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.315895 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.315733 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.315895 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.315757 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.315895 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.315781 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.315895 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.315797 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jmpr\" (UniqueName: \"kubernetes.io/projected/cf3016e2-a114-4579-b443-cb5429f0ce59-kube-api-access-6jmpr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.316253 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.316225 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.316352 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.316242 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.316352 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.316288 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.318049 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.318021 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf3016e2-a114-4579-b443-cb5429f0ce59-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.318197 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.318176 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf3016e2-a114-4579-b443-cb5429f0ce59-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.323404 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.323382 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jmpr\" (UniqueName: \"kubernetes.io/projected/cf3016e2-a114-4579-b443-cb5429f0ce59-kube-api-access-6jmpr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kkdng\" (UID: \"cf3016e2-a114-4579-b443-cb5429f0ce59\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.444156 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.444122 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:34.574807 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.574754 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng"] Jul 18 00:51:34.580145 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:34.580121 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 00:51:35.423912 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:35.423864 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" event={"ID":"cf3016e2-a114-4579-b443-cb5429f0ce59","Type":"ContainerStarted","Data":"2620e790da0d0be9689744cd9ef3fcd871e2dacc03f9287ffee6e77bcf33989f"} Jul 18 00:51:36.429574 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:36.429535 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" event={"ID":"cf3016e2-a114-4579-b443-cb5429f0ce59","Type":"ContainerStarted","Data":"0a8f1c3d9a94438a3e8fc83545d4a91bb2519664a42805e6f264105d0aaf1546"} Jul 18 00:51:36.446471 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:36.446423 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" podStartSLOduration=1.350172337 podStartE2EDuration="2.446409128s" totalImagesPullingTime="1.096236791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:51:36.445033696 +0000 UTC m=+667.467168631" watchObservedRunningTime="2026-07-18 00:51:36.446409128 +0000 UTC m=+667.468543933" Jul 18 00:51:39.410873 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:39.410824 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:44.445142 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:44.445080 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:44.445142 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:44.445159 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:44.457973 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:44.457942 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:44.472703 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:44.472679 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kkdng" Jul 18 00:51:48.753731 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:48.753687 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:58.746035 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.745996 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4"] Jul 18 00:51:58.749283 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.749264 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.751632 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.751607 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 18 00:51:58.758959 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.758934 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4"] Jul 18 00:51:58.842581 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.842541 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:51:58.842784 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.842760 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g85mg\" (UniqueName: \"kubernetes.io/projected/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-kube-api-access-g85mg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.842870 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.842795 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.842934 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.842880 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.842934 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.842915 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.843002 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.842973 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.843048 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.842999 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.943475 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.943437 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.943670 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.943486 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.943670 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.943643 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.943881 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.943700 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.943881 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.943781 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g85mg\" (UniqueName: \"kubernetes.io/projected/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-kube-api-access-g85mg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.943881 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.943815 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.944080 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.943881 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.944144 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.944118 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.944198 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.944184 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.945997 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.945974 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.946197 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.946178 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:58.951933 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:58.951907 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g85mg\" (UniqueName: \"kubernetes.io/projected/f19fcafc-9bf1-4f3b-b18b-2f9623874e04-kube-api-access-g85mg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4g8l4\" (UID: \"f19fcafc-9bf1-4f3b-b18b-2f9623874e04\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:59.059822 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:59.059736 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:51:59.192347 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:59.192323 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4"] Jul 18 00:51:59.195038 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:51:59.195012 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19fcafc_9bf1_4f3b_b18b_2f9623874e04.slice/crio-f80514959fdf85bc8f8c098c6e75fba6ad46b91e83d110db9861114d4b46653d WatchSource:0}: Error finding container f80514959fdf85bc8f8c098c6e75fba6ad46b91e83d110db9861114d4b46653d: Status 404 returned error can't find the container with id f80514959fdf85bc8f8c098c6e75fba6ad46b91e83d110db9861114d4b46653d Jul 18 00:51:59.525711 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:59.525668 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" event={"ID":"f19fcafc-9bf1-4f3b-b18b-2f9623874e04","Type":"ContainerStarted","Data":"3372acd7740473e097338b6401f20f3df4072ea30bcf1147202f2b9835128046"} Jul 18 00:51:59.525909 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:59.525737 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" event={"ID":"f19fcafc-9bf1-4f3b-b18b-2f9623874e04","Type":"ContainerStarted","Data":"f80514959fdf85bc8f8c098c6e75fba6ad46b91e83d110db9861114d4b46653d"} Jul 18 00:51:59.542113 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:51:59.542065 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" podStartSLOduration=1.385376484 podStartE2EDuration="1.542049472s" totalImagesPullingTime="156.672988ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:51:59.540528377 +0000 UTC m=+690.562663188" watchObservedRunningTime="2026-07-18 00:51:59.542049472 +0000 UTC m=+690.564184256" Jul 18 00:52:00.097930 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:00.097897 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:52:04.931553 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:04.931520 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk"] Jul 18 00:52:04.935244 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:04.935219 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:04.941708 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:04.941685 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 18 00:52:04.947931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:04.947902 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk"] Jul 18 00:52:04.966179 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:04.966135 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:52:05.002921 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.002886 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.002921 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.002921 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvm5d\" (UniqueName: \"kubernetes.io/projected/fe4642bd-237b-490d-a68d-afd7aa08d952-kube-api-access-nvm5d\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.003114 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.002953 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.003114 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.003027 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.003114 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.003088 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.003114 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.003106 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe4642bd-237b-490d-a68d-afd7aa08d952-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.103716 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.103677 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.103913 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.103731 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe4642bd-237b-490d-a68d-afd7aa08d952-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.103913 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.103798 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.103913 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.103854 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvm5d\" (UniqueName: \"kubernetes.io/projected/fe4642bd-237b-490d-a68d-afd7aa08d952-kube-api-access-nvm5d\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.103913 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.103906 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.104120 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.103950 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.104192 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.104169 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.104309 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.104289 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.104377 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.104363 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.106193 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.106163 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe4642bd-237b-490d-a68d-afd7aa08d952-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.106448 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.106426 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe4642bd-237b-490d-a68d-afd7aa08d952-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.111436 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.111413 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvm5d\" (UniqueName: \"kubernetes.io/projected/fe4642bd-237b-490d-a68d-afd7aa08d952-kube-api-access-nvm5d\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk\" (UID: \"fe4642bd-237b-490d-a68d-afd7aa08d952\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.245265 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.245170 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:05.380470 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.380445 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk"] Jul 18 00:52:05.382700 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:52:05.382674 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe4642bd_237b_490d_a68d_afd7aa08d952.slice/crio-109e1bb772763dccd4cf0202f8149d8b13d96944ed381c78f4a4f9ea9b7b1aef WatchSource:0}: Error finding container 109e1bb772763dccd4cf0202f8149d8b13d96944ed381c78f4a4f9ea9b7b1aef: Status 404 returned error can't find the container with id 109e1bb772763dccd4cf0202f8149d8b13d96944ed381c78f4a4f9ea9b7b1aef Jul 18 00:52:05.560619 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:05.560589 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" event={"ID":"fe4642bd-237b-490d-a68d-afd7aa08d952","Type":"ContainerStarted","Data":"109e1bb772763dccd4cf0202f8149d8b13d96944ed381c78f4a4f9ea9b7b1aef"} Jul 18 00:52:06.565929 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:06.565885 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" event={"ID":"fe4642bd-237b-490d-a68d-afd7aa08d952","Type":"ContainerStarted","Data":"0486ce0d4da48ecb2dbfc7232a282f3b8d6c9b7a2f5761efc8b5866d0bd4b6be"} Jul 18 00:52:06.583492 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:06.583447 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" podStartSLOduration=2.425362716 podStartE2EDuration="2.583429723s" totalImagesPullingTime="158.067007ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:52:06.581765127 +0000 UTC m=+697.603899911" watchObservedRunningTime="2026-07-18 00:52:06.583429723 +0000 UTC m=+697.605564507" Jul 18 00:52:09.060237 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:09.060195 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:52:09.060669 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:09.060275 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:52:09.072912 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:09.072887 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:52:09.590780 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:09.590693 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4g8l4" Jul 18 00:52:11.306239 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:11.306198 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:52:11.374991 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:11.374952 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:52:15.245443 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:15.245402 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:15.245443 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:15.245452 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:15.258467 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:15.258442 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:15.614887 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:15.614791 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk" Jul 18 00:52:36.959061 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:52:36.958960 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:53:10.125649 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.125609 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7784f8bd8-lvk92"] Jul 18 00:53:10.128946 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.128925 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7784f8bd8-lvk92" Jul 18 00:53:10.141762 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.141736 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7784f8bd8-lvk92"] Jul 18 00:53:10.197742 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.197703 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjkbz\" (UniqueName: \"kubernetes.io/projected/d35472df-0f65-47fa-afb2-e7a8f31e8fac-kube-api-access-sjkbz\") pod \"authorino-7784f8bd8-lvk92\" (UID: \"d35472df-0f65-47fa-afb2-e7a8f31e8fac\") " pod="kuadrant-system/authorino-7784f8bd8-lvk92" Jul 18 00:53:10.197742 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.197746 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d35472df-0f65-47fa-afb2-e7a8f31e8fac-tls-cert\") pod \"authorino-7784f8bd8-lvk92\" (UID: \"d35472df-0f65-47fa-afb2-e7a8f31e8fac\") " pod="kuadrant-system/authorino-7784f8bd8-lvk92" Jul 18 00:53:10.298505 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.298472 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjkbz\" (UniqueName: \"kubernetes.io/projected/d35472df-0f65-47fa-afb2-e7a8f31e8fac-kube-api-access-sjkbz\") pod \"authorino-7784f8bd8-lvk92\" (UID: \"d35472df-0f65-47fa-afb2-e7a8f31e8fac\") " pod="kuadrant-system/authorino-7784f8bd8-lvk92" Jul 18 00:53:10.298690 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.298514 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d35472df-0f65-47fa-afb2-e7a8f31e8fac-tls-cert\") pod \"authorino-7784f8bd8-lvk92\" (UID: \"d35472df-0f65-47fa-afb2-e7a8f31e8fac\") " pod="kuadrant-system/authorino-7784f8bd8-lvk92" Jul 18 00:53:10.301006 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.300973 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d35472df-0f65-47fa-afb2-e7a8f31e8fac-tls-cert\") pod \"authorino-7784f8bd8-lvk92\" (UID: \"d35472df-0f65-47fa-afb2-e7a8f31e8fac\") " pod="kuadrant-system/authorino-7784f8bd8-lvk92" Jul 18 00:53:10.306641 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.306619 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjkbz\" (UniqueName: \"kubernetes.io/projected/d35472df-0f65-47fa-afb2-e7a8f31e8fac-kube-api-access-sjkbz\") pod \"authorino-7784f8bd8-lvk92\" (UID: \"d35472df-0f65-47fa-afb2-e7a8f31e8fac\") " pod="kuadrant-system/authorino-7784f8bd8-lvk92" Jul 18 00:53:10.438437 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.438406 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7784f8bd8-lvk92" Jul 18 00:53:10.481773 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.481739 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:53:10.573438 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.573412 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7784f8bd8-lvk92"] Jul 18 00:53:10.576115 ip-10-0-135-148 kubenswrapper[2585]: W0718 00:53:10.576082 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd35472df_0f65_47fa_afb2_e7a8f31e8fac.slice/crio-e7f6ee5297baa822281fa0684cb222de81e96a06ecca27d9410a92836cc301fe WatchSource:0}: Error finding container e7f6ee5297baa822281fa0684cb222de81e96a06ecca27d9410a92836cc301fe: Status 404 returned error can't find the container with id e7f6ee5297baa822281fa0684cb222de81e96a06ecca27d9410a92836cc301fe Jul 18 00:53:10.576797 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.576705 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:53:10.855133 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:10.855054 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7784f8bd8-lvk92" event={"ID":"d35472df-0f65-47fa-afb2-e7a8f31e8fac","Type":"ContainerStarted","Data":"e7f6ee5297baa822281fa0684cb222de81e96a06ecca27d9410a92836cc301fe"} Jul 18 00:53:11.614965 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:11.614926 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:53:11.706124 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:11.706088 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:53:11.863277 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:11.863240 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7784f8bd8-lvk92" event={"ID":"d35472df-0f65-47fa-afb2-e7a8f31e8fac","Type":"ContainerStarted","Data":"9fc2b9d346f65edf05ceddee10656c35afdbb60c69bfc5f38ca85492fda0adb6"} Jul 18 00:53:11.884202 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:11.884103 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7784f8bd8-lvk92" podStartSLOduration=1.501949295 podStartE2EDuration="1.884085751s" totalImagesPullingTime="382.136456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 00:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 00:53:11.883149458 +0000 UTC m=+762.905284243" watchObservedRunningTime="2026-07-18 00:53:11.884085751 +0000 UTC m=+762.906220587" Jul 18 00:53:11.913821 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:11.913769 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-ff84948f7-kldkx"] Jul 18 00:53:11.914070 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:11.914028 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-ff84948f7-kldkx" podUID="9a158b93-a53e-42d9-b654-e29f00136e35" containerName="authorino" containerID="cri-o://c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a" gracePeriod=30 Jul 18 00:53:12.188702 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.188671 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff84948f7-kldkx" Jul 18 00:53:12.221561 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.221527 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a158b93-a53e-42d9-b654-e29f00136e35-tls-cert\") pod \"9a158b93-a53e-42d9-b654-e29f00136e35\" (UID: \"9a158b93-a53e-42d9-b654-e29f00136e35\") " Jul 18 00:53:12.221742 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.221630 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2rnx\" (UniqueName: \"kubernetes.io/projected/9a158b93-a53e-42d9-b654-e29f00136e35-kube-api-access-t2rnx\") pod \"9a158b93-a53e-42d9-b654-e29f00136e35\" (UID: \"9a158b93-a53e-42d9-b654-e29f00136e35\") " Jul 18 00:53:12.223931 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.223893 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a158b93-a53e-42d9-b654-e29f00136e35-kube-api-access-t2rnx" (OuterVolumeSpecName: "kube-api-access-t2rnx") pod "9a158b93-a53e-42d9-b654-e29f00136e35" (UID: "9a158b93-a53e-42d9-b654-e29f00136e35"). InnerVolumeSpecName "kube-api-access-t2rnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 00:53:12.232181 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.232158 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a158b93-a53e-42d9-b654-e29f00136e35-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9a158b93-a53e-42d9-b654-e29f00136e35" (UID: "9a158b93-a53e-42d9-b654-e29f00136e35"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 00:53:12.323027 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.322989 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2rnx\" (UniqueName: \"kubernetes.io/projected/9a158b93-a53e-42d9-b654-e29f00136e35-kube-api-access-t2rnx\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:53:12.323027 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.323022 2585 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9a158b93-a53e-42d9-b654-e29f00136e35-tls-cert\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 00:53:12.869132 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.869094 2585 generic.go:356] "Generic (PLEG): container finished" podID="9a158b93-a53e-42d9-b654-e29f00136e35" containerID="c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a" exitCode=0 Jul 18 00:53:12.869132 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.869136 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ff84948f7-kldkx" Jul 18 00:53:12.869643 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.869183 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff84948f7-kldkx" event={"ID":"9a158b93-a53e-42d9-b654-e29f00136e35","Type":"ContainerDied","Data":"c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a"} Jul 18 00:53:12.869643 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.869229 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ff84948f7-kldkx" event={"ID":"9a158b93-a53e-42d9-b654-e29f00136e35","Type":"ContainerDied","Data":"074a58749ef09637b35c1ba9318e53c1a9f94b8e0acaad2f7e13cb6f374ba3cf"} Jul 18 00:53:12.869643 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.869256 2585 scope.go:117] "RemoveContainer" containerID="c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a" Jul 18 00:53:12.879513 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.879493 2585 scope.go:117] "RemoveContainer" containerID="c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a" Jul 18 00:53:12.879803 ip-10-0-135-148 kubenswrapper[2585]: E0718 00:53:12.879784 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a\": container with ID starting with c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a not found: ID does not exist" containerID="c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a" Jul 18 00:53:12.879901 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.879810 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a"} err="failed to get container status \"c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a\": rpc error: code = NotFound desc = could not find container \"c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a\": container with ID starting with c358febacd6a1a81ca6f37ac095d3b8cbc460f5a6090393478bbdbafb0a12c0a not found: ID does not exist" Jul 18 00:53:12.891017 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.890992 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-ff84948f7-kldkx"] Jul 18 00:53:12.894856 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:12.894804 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-ff84948f7-kldkx"] Jul 18 00:53:13.456426 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:13.456391 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a158b93-a53e-42d9-b654-e29f00136e35" path="/var/lib/kubelet/pods/9a158b93-a53e-42d9-b654-e29f00136e35/volumes" Jul 18 00:53:35.486643 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:35.486606 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:53:35.588752 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:53:35.588717 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:55:15.203546 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:55:15.203499 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:55:16.415360 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:55:16.415327 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:55:29.386978 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:55:29.386899 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 00:55:29.388977 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:55:29.388947 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 00:55:29.391326 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:55:29.391305 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 00:55:29.392886 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:55:29.392862 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 00:55:33.176958 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:55:33.176926 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:55:33.278018 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:55:33.277985 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:55:58.893968 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:55:58.893851 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:56:00.063222 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:56:00.063187 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:56:04.859807 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:56:04.859773 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:56:04.962787 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:56:04.962751 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:56:05.108907 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:56:05.108873 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:56:05.223438 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:56:05.223403 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:56:10.998473 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:56:10.998436 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:56:11.061621 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:56:11.061559 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 00:59:44.130626 ip-10-0-135-148 kubenswrapper[2585]: I0718 00:59:44.130589 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:00:04.590660 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:00:04.590572 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:00:29.413399 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:00:29.413369 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:00:29.414693 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:00:29.414662 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:00:29.417512 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:00:29.417492 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:00:29.418701 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:00:29.418683 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:00:50.887789 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:00:50.887748 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:05:29.444644 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:05:29.444613 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:05:29.445270 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:05:29.445078 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:05:29.449049 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:05:29.449027 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:05:29.449544 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:05:29.449524 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:06:51.488839 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:06:51.488804 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:06:53.218619 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:06:53.218568 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:09.656031 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:09.655997 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:19.238396 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:19.238343 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:27.493210 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:27.493133 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:32.565698 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:32.565655 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:35.449646 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:35.449601 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:35.547954 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:35.547912 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:35.877967 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:35.877879 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:41.748574 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:41.748537 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:41.823245 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:41.823210 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:42.032866 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:42.032764 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:42.165445 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:42.165416 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:43.185930 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:43.185895 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:43.280262 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:43.280226 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:43.347977 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:43.347937 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:43.757780 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:43.757745 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:43.860045 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:43.860001 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:43.959390 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:43.959355 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:07:45.873074 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:07:45.873039 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:08:30.085360 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:08:30.085320 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:08:30.133021 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:08:30.132990 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:10:29.472948 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:29.472910 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:10:29.473446 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:29.473425 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:10:29.476980 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:29.476954 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:10:29.477388 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:29.477373 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:10:40.599472 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:40.599432 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:10:40.687075 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:40.687024 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:10:47.548549 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:47.548518 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:10:47.650227 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:47.650193 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:10:48.329660 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:48.329630 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:10:48.421439 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:48.421405 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:10:54.294587 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:54.294549 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:10:54.456149 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:54.456112 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:10:59.843485 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:10:59.843453 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:00.185665 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:00.185626 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:00.285962 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:00.285914 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:00.452485 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:00.452405 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:00.556818 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:00.556781 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:00.652707 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:00.652670 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:00.752950 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:00.752865 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:01.870191 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:01.870160 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:01.984940 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:01.984906 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:02.083487 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:02.083455 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:02.462772 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:02.462736 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:02.559552 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:02.559511 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:11:04.001157 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:11:04.001112 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:06.503101 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:06.503054 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:07.904798 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:07.904751 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:11.792594 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:11.792553 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:12.450245 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:12.450210 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:13.121894 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:13.121861 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:17.980551 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:17.980519 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:18.081031 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:18.080995 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:18.205655 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:18.205620 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:18.301062 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:18.300978 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:19.314822 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:19.314788 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:19.404208 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:19.404180 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:19.475071 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:19.475046 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:19.852723 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:19.852688 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:20.033753 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:20.033718 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:20.134289 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:20.134206 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:12:21.789412 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:12:21.789378 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:15:29.508196 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:15:29.508171 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:15:29.509399 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:15:29.509368 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:15:29.512312 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:15:29.512289 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:15:29.513719 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:15:29.513701 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:15:41.353749 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:15:41.353714 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:15:41.438449 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:15:41.438411 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:15:44.520874 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:15:44.520816 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:15:45.744601 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:15:45.744564 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:15:45.844972 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:15:45.844941 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:16:12.626988 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:16:12.626943 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:17:51.972570 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:17:51.972529 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:24.696241 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:24.696206 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:24.797500 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:24.797430 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:29.963055 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:29.963022 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:30.497085 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:30.497053 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:35.944692 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:35.944605 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:36.045278 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:36.045237 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:36.244557 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:36.244518 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:37.352531 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:37.352490 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:37.433953 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:37.433919 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:37.496592 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:37.496552 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:37.766727 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:37.766688 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:38.120506 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:38.120419 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:38.221129 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:38.221091 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:46.436468 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:46.436424 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:46.536875 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:46.536838 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:18:50.380014 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:18:50.379987 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:20:29.533665 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:20:29.533631 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:20:29.542095 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:20:29.542065 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:20:29.543681 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:20:29.543651 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:20:29.545711 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:20:29.545692 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:20:42.074173 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:20:42.074140 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:20:42.164387 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:20:42.164346 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:20:52.445546 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:20:52.445505 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:20:52.546474 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:20:52.546434 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:21:41.978992 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:21:41.978947 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:21:42.078947 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:21:42.078903 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:21:47.160541 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:21:47.160493 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:21:47.752092 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:21:47.752037 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:21:53.276348 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:21:53.276304 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:21:53.377502 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:21:53.377463 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:21:53.778008 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:21:53.777970 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:21:53.878212 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:21:53.878179 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:21:53.991906 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:21:53.991869 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:21:54.130465 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:21:54.130370 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:21:55.490470 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:21:55.490435 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:01.282871 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:01.282820 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:01.391693 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:01.391659 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:01.483857 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:01.483807 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:01.583649 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:01.583565 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:02.056372 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:02.056339 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:28.053176 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:28.053085 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:33.912795 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:33.912752 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:34.019733 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:34.019690 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:34.112930 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:34.112897 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:50.004248 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:50.004208 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:54.898687 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:54.898655 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:55.892441 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:55.892407 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:56.055465 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:56.055429 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:22:56.188324 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:22:56.188279 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:06.618268 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:06.618230 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:07.479130 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:07.479090 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:13.054313 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:13.054272 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:13.109229 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:13.109191 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:13.204188 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:13.204150 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:13.303638 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:13.303602 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:13.403292 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:13.403261 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:14.192072 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:14.192030 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:14.288149 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:14.288116 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:14.388720 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:14.388682 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:16.078617 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:16.078581 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:21.840909 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:21.840869 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:21.939762 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:21.939727 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:22.535168 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:22.535140 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:28.423570 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:28.423534 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:28.521289 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:28.521236 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:28.660485 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:28.660448 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:28.964815 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:28.964781 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:29.618434 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:29.618399 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:32.388887 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:32.388811 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:32.470387 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:32.470351 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:32.557339 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:32.557305 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:33.143745 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:33.143715 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:33.242935 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:33.242883 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:33.326809 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:33.326775 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:36.866905 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:36.866860 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:37.541216 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:37.541170 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:37.639294 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:37.639251 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:37.970701 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:37.970667 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:43.821901 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:43.821854 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:43.885962 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:43.885925 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:54.870512 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:54.870454 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:23:59.254940 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:23:59.254897 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:04.937393 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:04.937356 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:05.038253 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:05.038220 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:09.946302 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:09.946267 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:10.193578 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:10.193540 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:10.313222 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:10.313130 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:10.413286 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:10.413254 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:10.536265 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:10.536232 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:16.299066 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:16.299030 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:16.406599 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:16.406560 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:16.502213 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:16.502182 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:16.679225 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:16.679190 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:22.320763 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:22.320713 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:22.382364 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:22.382326 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:29.212453 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:29.212419 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:29.621087 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:29.620997 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:29.819321 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:29.819280 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:32.721037 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:32.720993 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:32.834358 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:32.834316 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:32.933239 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:32.933201 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:33.475647 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:33.475606 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:33.576043 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:33.575990 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:33.676320 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:33.676284 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:37.221785 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:37.221743 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:42.837187 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.837144 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn"] Jul 18 01:24:42.837692 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.837625 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a158b93-a53e-42d9-b654-e29f00136e35" containerName="authorino" Jul 18 01:24:42.837692 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.837644 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a158b93-a53e-42d9-b654-e29f00136e35" containerName="authorino" Jul 18 01:24:42.837810 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.837747 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a158b93-a53e-42d9-b654-e29f00136e35" containerName="authorino" Jul 18 01:24:42.840673 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.840650 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.843379 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.843353 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-46dd73e1\"/\"kube-root-ca.crt\"" Jul 18 01:24:42.844279 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.844261 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-46dd73e1\"/\"openshift-service-ca.crt\"" Jul 18 01:24:42.844528 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.844510 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-46dd73e1\"/\"default-dockercfg-cdz6h\"" Jul 18 01:24:42.844631 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.844514 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-46dd73e1\"/\"e2e-lifecycle-model-46dd73e1-kserve-self-signed-certs\"" Jul 18 01:24:42.850792 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.850767 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-tmp-dir\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.850950 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.850816 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn"] Jul 18 01:24:42.850950 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.850821 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-dshm\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.850950 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.850909 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-model-cache\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.851079 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.850956 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-home\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.851116 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.851096 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1773663f-fcc9-419f-a10a-a4e674529988-tls-certs\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.851150 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.851138 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5frb\" (UniqueName: \"kubernetes.io/projected/1773663f-fcc9-419f-a10a-a4e674529988-kube-api-access-w5frb\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.874820 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.874783 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:42.951964 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.951927 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-tmp-dir\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.952168 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.951983 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-dshm\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.952168 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.952015 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-model-cache\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.952168 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.952038 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-home\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.952168 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.952088 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1773663f-fcc9-419f-a10a-a4e674529988-tls-certs\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.952370 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.952219 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5frb\" (UniqueName: \"kubernetes.io/projected/1773663f-fcc9-419f-a10a-a4e674529988-kube-api-access-w5frb\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.952444 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.952417 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-model-cache\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.952504 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.952414 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-tmp-dir\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.952564 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.952525 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-home\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.954608 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.954571 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-dshm\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.954750 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.954685 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1773663f-fcc9-419f-a10a-a4e674529988-tls-certs\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:42.960119 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:42.960096 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5frb\" (UniqueName: \"kubernetes.io/projected/1773663f-fcc9-419f-a10a-a4e674529988-kube-api-access-w5frb\") pod \"e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:43.029348 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:43.029304 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:43.081531 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:43.081493 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:24:43.150898 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:43.150865 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:43.288228 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:43.288196 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn"] Jul 18 01:24:43.290889 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:24:43.290861 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1773663f_fcc9_419f_a10a_a4e674529988.slice/crio-59e4fdb575fc59a4443a93af4426041506c9ccd488361ecfdbbaa3158a288f26 WatchSource:0}: Error finding container 59e4fdb575fc59a4443a93af4426041506c9ccd488361ecfdbbaa3158a288f26: Status 404 returned error can't find the container with id 59e4fdb575fc59a4443a93af4426041506c9ccd488361ecfdbbaa3158a288f26 Jul 18 01:24:44.240004 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:44.239954 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" event={"ID":"1773663f-fcc9-419f-a10a-a4e674529988","Type":"ContainerStarted","Data":"f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b"} Jul 18 01:24:44.240004 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:44.240009 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" event={"ID":"1773663f-fcc9-419f-a10a-a4e674529988","Type":"ContainerStarted","Data":"59e4fdb575fc59a4443a93af4426041506c9ccd488361ecfdbbaa3158a288f26"} Jul 18 01:24:44.257210 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:44.257157 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" podStartSLOduration=2.257137619 podStartE2EDuration="2.257137619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:24:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:24:44.255376883 +0000 UTC m=+2655.277511681" watchObservedRunningTime="2026-07-18 01:24:44.257137619 +0000 UTC m=+2655.279272403" Jul 18 01:24:53.151418 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:53.151358 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:53.151418 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:53.151437 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:53.164717 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:53.164685 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:24:53.293882 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:24:53.293825 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:25:20.960477 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:20.960431 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:25:25.571575 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:25.571489 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:25:29.565569 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:29.565537 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:25:29.569451 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:29.569414 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:25:29.569783 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:29.569766 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:25:29.573907 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:29.573884 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:25:30.561063 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:30.561031 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn"] Jul 18 01:25:30.656039 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:30.655998 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:25:42.796604 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:42.796569 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:25:42.895767 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:42.895731 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:25:45.582216 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.582169 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" podUID="1773663f-fcc9-419f-a10a-a4e674529988" containerName="main" containerID="cri-o://f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b" gracePeriod=45 Jul 18 01:25:45.832692 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.832625 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:25:45.904959 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.904927 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5frb\" (UniqueName: \"kubernetes.io/projected/1773663f-fcc9-419f-a10a-a4e674529988-kube-api-access-w5frb\") pod \"1773663f-fcc9-419f-a10a-a4e674529988\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " Jul 18 01:25:45.905150 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.904970 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-model-cache\") pod \"1773663f-fcc9-419f-a10a-a4e674529988\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " Jul 18 01:25:45.905150 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.905019 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-tmp-dir\") pod \"1773663f-fcc9-419f-a10a-a4e674529988\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " Jul 18 01:25:45.905150 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.905054 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-dshm\") pod \"1773663f-fcc9-419f-a10a-a4e674529988\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " Jul 18 01:25:45.905150 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.905081 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1773663f-fcc9-419f-a10a-a4e674529988-tls-certs\") pod \"1773663f-fcc9-419f-a10a-a4e674529988\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " Jul 18 01:25:45.905150 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.905112 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-home\") pod \"1773663f-fcc9-419f-a10a-a4e674529988\" (UID: \"1773663f-fcc9-419f-a10a-a4e674529988\") " Jul 18 01:25:45.905421 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.905300 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1773663f-fcc9-419f-a10a-a4e674529988" (UID: "1773663f-fcc9-419f-a10a-a4e674529988"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:25:45.905421 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.905310 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-model-cache" (OuterVolumeSpecName: "model-cache") pod "1773663f-fcc9-419f-a10a-a4e674529988" (UID: "1773663f-fcc9-419f-a10a-a4e674529988"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:25:45.905532 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.905445 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-home" (OuterVolumeSpecName: "home") pod "1773663f-fcc9-419f-a10a-a4e674529988" (UID: "1773663f-fcc9-419f-a10a-a4e674529988"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:25:45.905699 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.905676 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-model-cache\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:25:45.905699 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.905700 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-tmp-dir\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:25:45.905894 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.905712 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-home\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:25:45.907666 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.907640 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1773663f-fcc9-419f-a10a-a4e674529988-kube-api-access-w5frb" (OuterVolumeSpecName: "kube-api-access-w5frb") pod "1773663f-fcc9-419f-a10a-a4e674529988" (UID: "1773663f-fcc9-419f-a10a-a4e674529988"). InnerVolumeSpecName "kube-api-access-w5frb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:25:45.907666 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.907648 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-dshm" (OuterVolumeSpecName: "dshm") pod "1773663f-fcc9-419f-a10a-a4e674529988" (UID: "1773663f-fcc9-419f-a10a-a4e674529988"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:25:45.907794 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:45.907679 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1773663f-fcc9-419f-a10a-a4e674529988-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1773663f-fcc9-419f-a10a-a4e674529988" (UID: "1773663f-fcc9-419f-a10a-a4e674529988"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:25:46.006500 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.006464 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1773663f-fcc9-419f-a10a-a4e674529988-dshm\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:25:46.006500 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.006493 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1773663f-fcc9-419f-a10a-a4e674529988-tls-certs\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:25:46.006500 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.006506 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5frb\" (UniqueName: \"kubernetes.io/projected/1773663f-fcc9-419f-a10a-a4e674529988-kube-api-access-w5frb\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:25:46.528600 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.528557 2585 generic.go:356] "Generic (PLEG): container finished" podID="1773663f-fcc9-419f-a10a-a4e674529988" containerID="f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b" exitCode=0 Jul 18 01:25:46.528786 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.528615 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" Jul 18 01:25:46.528786 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.528636 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" event={"ID":"1773663f-fcc9-419f-a10a-a4e674529988","Type":"ContainerDied","Data":"f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b"} Jul 18 01:25:46.528786 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.528679 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn" event={"ID":"1773663f-fcc9-419f-a10a-a4e674529988","Type":"ContainerDied","Data":"59e4fdb575fc59a4443a93af4426041506c9ccd488361ecfdbbaa3158a288f26"} Jul 18 01:25:46.528786 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.528697 2585 scope.go:117] "RemoveContainer" containerID="f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b" Jul 18 01:25:46.538139 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.538117 2585 scope.go:117] "RemoveContainer" containerID="f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b" Jul 18 01:25:46.538472 ip-10-0-135-148 kubenswrapper[2585]: E0718 01:25:46.538451 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b\": container with ID starting with f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b not found: ID does not exist" containerID="f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b" Jul 18 01:25:46.538524 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.538483 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b"} err="failed to get container status \"f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b\": rpc error: code = NotFound desc = could not find container \"f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b\": container with ID starting with f298e41f944e14215d3323da05181e843eead1b9edf2f092f3a844873255a22b not found: ID does not exist" Jul 18 01:25:46.550157 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.550127 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn"] Jul 18 01:25:46.554201 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:46.554172 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-46dd73e1/e2e-lifecycle-model-46dd73e1-kserve-7896968659-b48qn"] Jul 18 01:25:47.457011 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:25:47.456966 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1773663f-fcc9-419f-a10a-a4e674529988" path="/var/lib/kubelet/pods/1773663f-fcc9-419f-a10a-a4e674529988/volumes" Jul 18 01:26:06.695453 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:06.695410 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:07.372121 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:07.372086 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:12.961695 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:12.961660 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:13.029046 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:13.029004 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:17.751494 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:17.751459 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:17.950151 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:17.950110 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:18.951041 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:18.951006 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:19.048967 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:19.048936 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:19.152801 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:19.152769 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:19.819642 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:19.819605 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:19.918975 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:19.918944 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:20.019177 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:20.019141 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:21.308220 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:21.308182 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:26.973232 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:26.973198 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:27.073662 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:27.073629 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:27.623346 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:27.623313 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:33.406700 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:33.406660 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:33.468435 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:33.468400 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:33.556013 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:33.555966 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:33.654754 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:33.654714 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:44.130986 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:44.130944 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:26:44.443393 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:26:44.443356 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:05.380401 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:05.380363 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:10.953278 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:10.953238 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:11.010595 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:11.010561 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:11.329072 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:11.328981 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:11.618758 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:11.618669 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:11.825039 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:11.825005 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:14.638450 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:14.638413 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:14.721986 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:14.721935 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:14.791458 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:14.791418 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:15.098213 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:15.098170 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:15.396548 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:15.396515 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:15.483421 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:15.483389 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:19.126801 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:19.126765 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:19.459236 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:19.459198 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:19.756511 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:19.756425 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:19.948178 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:19.948136 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:22.681802 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:22.681765 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:22.768572 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:22.768538 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:22.840979 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:22.840936 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:22.937989 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:22.937902 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:23.400589 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:23.400559 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:23.523055 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:23.523018 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:26.318127 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:26.318089 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:27.164630 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:27.164598 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:36.288731 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:36.288687 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:36.389249 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:36.389214 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:45.104054 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:45.104009 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:45.201505 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:45.201472 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:46.436164 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.436129 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb"] Jul 18 01:27:46.436551 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.436439 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1773663f-fcc9-419f-a10a-a4e674529988" containerName="main" Jul 18 01:27:46.436551 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.436450 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1773663f-fcc9-419f-a10a-a4e674529988" containerName="main" Jul 18 01:27:46.436551 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.436523 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1773663f-fcc9-419f-a10a-a4e674529988" containerName="main" Jul 18 01:27:46.439381 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.439362 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.441716 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.441692 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-390947\"/\"openshift-service-ca.crt\"" Jul 18 01:27:46.441890 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.441728 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-390947\"/\"test-model-390947-kserve-self-signed-certs\"" Jul 18 01:27:46.442673 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.442649 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-390947\"/\"default-dockercfg-v6lhj\"" Jul 18 01:27:46.442673 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.442662 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-390947\"/\"kube-root-ca.crt\"" Jul 18 01:27:46.448722 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.448699 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb"] Jul 18 01:27:46.536272 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.536214 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:27:46.575252 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.575208 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-dshm\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.575252 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.575248 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-model-cache\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.575468 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.575273 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tls-certs\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.575553 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.575525 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tmp-dir\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.575608 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.575592 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lznv\" (UniqueName: \"kubernetes.io/projected/3aeb9a35-556d-4c12-a58f-5bd543c2939c-kube-api-access-7lznv\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.575670 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.575657 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-home\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.676458 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.676414 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tmp-dir\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.676458 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.676465 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lznv\" (UniqueName: \"kubernetes.io/projected/3aeb9a35-556d-4c12-a58f-5bd543c2939c-kube-api-access-7lznv\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.676723 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.676498 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-home\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.676723 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.676531 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-dshm\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.676723 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.676546 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-model-cache\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.676723 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.676570 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tls-certs\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.677018 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.676894 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tmp-dir\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.677018 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.676928 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-home\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.677121 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.677057 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-model-cache\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.678949 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.678932 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-dshm\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.679082 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.679065 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tls-certs\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.684903 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.684883 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lznv\" (UniqueName: \"kubernetes.io/projected/3aeb9a35-556d-4c12-a58f-5bd543c2939c-kube-api-access-7lznv\") pod \"test-model-390947-kserve-84c9654d49-dpnqb\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.750666 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.750579 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:46.891942 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:46.891913 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb"] Jul 18 01:27:46.894647 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:27:46.894614 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3aeb9a35_556d_4c12_a58f_5bd543c2939c.slice/crio-f4135be26542b197bb4684d224b79e642227095515ebdfbb38301bb1d258b1ce WatchSource:0}: Error finding container f4135be26542b197bb4684d224b79e642227095515ebdfbb38301bb1d258b1ce: Status 404 returned error can't find the container with id f4135be26542b197bb4684d224b79e642227095515ebdfbb38301bb1d258b1ce Jul 18 01:27:47.046602 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:47.046509 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" event={"ID":"3aeb9a35-556d-4c12-a58f-5bd543c2939c","Type":"ContainerStarted","Data":"1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f"} Jul 18 01:27:47.046602 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:47.046551 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" event={"ID":"3aeb9a35-556d-4c12-a58f-5bd543c2939c","Type":"ContainerStarted","Data":"f4135be26542b197bb4684d224b79e642227095515ebdfbb38301bb1d258b1ce"} Jul 18 01:27:47.064492 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:47.064437 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" podStartSLOduration=1.064417271 podStartE2EDuration="1.064417271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:27:47.062406125 +0000 UTC m=+2838.084540902" watchObservedRunningTime="2026-07-18 01:27:47.064417271 +0000 UTC m=+2838.086552055" Jul 18 01:27:56.751030 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:56.750987 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:56.751519 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:56.751051 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:56.763729 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:56.763701 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:27:57.100760 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:27:57.100667 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:28:11.732688 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.732651 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp"] Jul 18 01:28:11.735578 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.735559 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.739979 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.739958 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0a3406\"/\"test-model-0a3406-kserve-self-signed-certs\"" Jul 18 01:28:11.740188 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.739965 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0a3406\"/\"openshift-service-ca.crt\"" Jul 18 01:28:11.740188 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.739966 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0a3406\"/\"default-dockercfg-vln8d\"" Jul 18 01:28:11.740188 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.740077 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0a3406\"/\"kube-root-ca.crt\"" Jul 18 01:28:11.748185 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.748163 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp"] Jul 18 01:28:11.762681 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.762642 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:11.808397 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.808356 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ab84c23-568c-43b0-8b49-597a60b39e2c-tls-certs\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.808605 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.808441 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm9f6\" (UniqueName: \"kubernetes.io/projected/2ab84c23-568c-43b0-8b49-597a60b39e2c-kube-api-access-hm9f6\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.808605 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.808513 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-model-cache\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.808605 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.808536 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-tmp-dir\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.808605 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.808561 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-dshm\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.808780 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.808604 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-home\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.909270 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.909227 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-home\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.909465 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.909307 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ab84c23-568c-43b0-8b49-597a60b39e2c-tls-certs\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.909465 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.909373 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm9f6\" (UniqueName: \"kubernetes.io/projected/2ab84c23-568c-43b0-8b49-597a60b39e2c-kube-api-access-hm9f6\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.909465 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.909424 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-model-cache\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.909465 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.909448 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-tmp-dir\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.909677 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.909476 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-dshm\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.909817 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.909790 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-home\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.909934 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.909861 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-model-cache\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.909996 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.909938 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-tmp-dir\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.911672 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.911645 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-dshm\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.911797 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.911767 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ab84c23-568c-43b0-8b49-597a60b39e2c-tls-certs\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:11.917771 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:11.917748 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm9f6\" (UniqueName: \"kubernetes.io/projected/2ab84c23-568c-43b0-8b49-597a60b39e2c-kube-api-access-hm9f6\") pod \"test-model-0a3406-kserve-778bc49dd9-849gp\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:12.046077 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:12.045983 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:12.183129 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:12.182977 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp"] Jul 18 01:28:12.185846 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:28:12.185799 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ab84c23_568c_43b0_8b49_597a60b39e2c.slice/crio-eaea8a34f1c387ddd9bc04442196cca15f30082dbe2ffd2f0129cba00e19e060 WatchSource:0}: Error finding container eaea8a34f1c387ddd9bc04442196cca15f30082dbe2ffd2f0129cba00e19e060: Status 404 returned error can't find the container with id eaea8a34f1c387ddd9bc04442196cca15f30082dbe2ffd2f0129cba00e19e060 Jul 18 01:28:13.163132 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:13.163090 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" event={"ID":"2ab84c23-568c-43b0-8b49-597a60b39e2c","Type":"ContainerStarted","Data":"117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1"} Jul 18 01:28:13.163553 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:13.163148 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" event={"ID":"2ab84c23-568c-43b0-8b49-597a60b39e2c","Type":"ContainerStarted","Data":"eaea8a34f1c387ddd9bc04442196cca15f30082dbe2ffd2f0129cba00e19e060"} Jul 18 01:28:13.180904 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:13.180820 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" podStartSLOduration=2.180803198 podStartE2EDuration="2.180803198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:28:13.17829604 +0000 UTC m=+2864.200430825" watchObservedRunningTime="2026-07-18 01:28:13.180803198 +0000 UTC m=+2864.202937982" Jul 18 01:28:22.047123 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:22.047019 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:22.047123 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:22.047094 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:22.059909 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:22.059876 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:22.216788 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:22.216751 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:30.629617 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:30.629580 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb"] Jul 18 01:28:30.649250 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:30.649216 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:31.640502 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:31.640461 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp"] Jul 18 01:28:31.657559 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:31.657523 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:32.270721 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:32.270681 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:32.369367 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:32.369328 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:42.382617 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:42.382567 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:45.650577 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.650535 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" podUID="3aeb9a35-556d-4c12-a58f-5bd543c2939c" containerName="main" containerID="cri-o://1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f" gracePeriod=45 Jul 18 01:28:45.902559 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.902487 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:28:45.933272 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.933237 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-model-cache\") pod \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " Jul 18 01:28:45.933272 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.933278 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-dshm\") pod \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " Jul 18 01:28:45.933532 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.933309 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lznv\" (UniqueName: \"kubernetes.io/projected/3aeb9a35-556d-4c12-a58f-5bd543c2939c-kube-api-access-7lznv\") pod \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " Jul 18 01:28:45.933532 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.933394 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-home\") pod \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " Jul 18 01:28:45.933532 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.933428 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tls-certs\") pod \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " Jul 18 01:28:45.933532 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.933464 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tmp-dir\") pod \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\" (UID: \"3aeb9a35-556d-4c12-a58f-5bd543c2939c\") " Jul 18 01:28:45.933762 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.933521 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-model-cache" (OuterVolumeSpecName: "model-cache") pod "3aeb9a35-556d-4c12-a58f-5bd543c2939c" (UID: "3aeb9a35-556d-4c12-a58f-5bd543c2939c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:28:45.933762 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.933684 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-home" (OuterVolumeSpecName: "home") pod "3aeb9a35-556d-4c12-a58f-5bd543c2939c" (UID: "3aeb9a35-556d-4c12-a58f-5bd543c2939c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:28:45.933762 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.933740 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3aeb9a35-556d-4c12-a58f-5bd543c2939c" (UID: "3aeb9a35-556d-4c12-a58f-5bd543c2939c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:28:45.933762 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.933760 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-model-cache\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:45.935644 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.935620 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-dshm" (OuterVolumeSpecName: "dshm") pod "3aeb9a35-556d-4c12-a58f-5bd543c2939c" (UID: "3aeb9a35-556d-4c12-a58f-5bd543c2939c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:28:45.936142 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.936119 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3aeb9a35-556d-4c12-a58f-5bd543c2939c" (UID: "3aeb9a35-556d-4c12-a58f-5bd543c2939c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:28:45.936226 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:45.936134 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aeb9a35-556d-4c12-a58f-5bd543c2939c-kube-api-access-7lznv" (OuterVolumeSpecName: "kube-api-access-7lznv") pod "3aeb9a35-556d-4c12-a58f-5bd543c2939c" (UID: "3aeb9a35-556d-4c12-a58f-5bd543c2939c"). InnerVolumeSpecName "kube-api-access-7lznv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:28:46.034769 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.034728 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-home\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:46.034769 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.034763 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tls-certs\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:46.034769 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.034775 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-tmp-dir\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:46.035040 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.034788 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aeb9a35-556d-4c12-a58f-5bd543c2939c-dshm\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:46.035040 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.034801 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lznv\" (UniqueName: \"kubernetes.io/projected/3aeb9a35-556d-4c12-a58f-5bd543c2939c-kube-api-access-7lznv\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:46.317114 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.317025 2585 generic.go:356] "Generic (PLEG): container finished" podID="3aeb9a35-556d-4c12-a58f-5bd543c2939c" containerID="1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f" exitCode=0 Jul 18 01:28:46.317114 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.317068 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" Jul 18 01:28:46.317305 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.317117 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" event={"ID":"3aeb9a35-556d-4c12-a58f-5bd543c2939c","Type":"ContainerDied","Data":"1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f"} Jul 18 01:28:46.317305 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.317160 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb" event={"ID":"3aeb9a35-556d-4c12-a58f-5bd543c2939c","Type":"ContainerDied","Data":"f4135be26542b197bb4684d224b79e642227095515ebdfbb38301bb1d258b1ce"} Jul 18 01:28:46.317305 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.317178 2585 scope.go:117] "RemoveContainer" containerID="1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f" Jul 18 01:28:46.326108 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.326087 2585 scope.go:117] "RemoveContainer" containerID="1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f" Jul 18 01:28:46.326418 ip-10-0-135-148 kubenswrapper[2585]: E0718 01:28:46.326398 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f\": container with ID starting with 1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f not found: ID does not exist" containerID="1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f" Jul 18 01:28:46.326457 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.326427 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f"} err="failed to get container status \"1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f\": rpc error: code = NotFound desc = could not find container \"1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f\": container with ID starting with 1bb4927e10a05443193494813956142a326388148b079a2c9d12788ad72b2d9f not found: ID does not exist" Jul 18 01:28:46.338041 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.337978 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb"] Jul 18 01:28:46.340136 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.340109 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-390947/test-model-390947-kserve-84c9654d49-dpnqb"] Jul 18 01:28:46.661839 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.661774 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" podUID="2ab84c23-568c-43b0-8b49-597a60b39e2c" containerName="main" containerID="cri-o://117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1" gracePeriod=45 Jul 18 01:28:46.841076 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.841039 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:46.916597 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.916534 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:46.942788 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.942752 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm9f6\" (UniqueName: \"kubernetes.io/projected/2ab84c23-568c-43b0-8b49-597a60b39e2c-kube-api-access-hm9f6\") pod \"2ab84c23-568c-43b0-8b49-597a60b39e2c\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " Jul 18 01:28:46.942977 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.942794 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-dshm\") pod \"2ab84c23-568c-43b0-8b49-597a60b39e2c\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " Jul 18 01:28:46.942977 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.942859 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-tmp-dir\") pod \"2ab84c23-568c-43b0-8b49-597a60b39e2c\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " Jul 18 01:28:46.942977 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.942886 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-model-cache\") pod \"2ab84c23-568c-43b0-8b49-597a60b39e2c\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " Jul 18 01:28:46.942977 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.942947 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ab84c23-568c-43b0-8b49-597a60b39e2c-tls-certs\") pod \"2ab84c23-568c-43b0-8b49-597a60b39e2c\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " Jul 18 01:28:46.943182 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.943000 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-home\") pod \"2ab84c23-568c-43b0-8b49-597a60b39e2c\" (UID: \"2ab84c23-568c-43b0-8b49-597a60b39e2c\") " Jul 18 01:28:46.943182 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.943162 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-model-cache" (OuterVolumeSpecName: "model-cache") pod "2ab84c23-568c-43b0-8b49-597a60b39e2c" (UID: "2ab84c23-568c-43b0-8b49-597a60b39e2c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:28:46.943293 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.943177 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2ab84c23-568c-43b0-8b49-597a60b39e2c" (UID: "2ab84c23-568c-43b0-8b49-597a60b39e2c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:28:46.943400 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.943371 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-tmp-dir\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:46.943400 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.943384 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-home" (OuterVolumeSpecName: "home") pod "2ab84c23-568c-43b0-8b49-597a60b39e2c" (UID: "2ab84c23-568c-43b0-8b49-597a60b39e2c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:28:46.943400 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.943396 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-model-cache\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:46.945178 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.945152 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-dshm" (OuterVolumeSpecName: "dshm") pod "2ab84c23-568c-43b0-8b49-597a60b39e2c" (UID: "2ab84c23-568c-43b0-8b49-597a60b39e2c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:28:46.945297 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.945239 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ab84c23-568c-43b0-8b49-597a60b39e2c-kube-api-access-hm9f6" (OuterVolumeSpecName: "kube-api-access-hm9f6") pod "2ab84c23-568c-43b0-8b49-597a60b39e2c" (UID: "2ab84c23-568c-43b0-8b49-597a60b39e2c"). InnerVolumeSpecName "kube-api-access-hm9f6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:28:46.945389 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:46.945376 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ab84c23-568c-43b0-8b49-597a60b39e2c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2ab84c23-568c-43b0-8b49-597a60b39e2c" (UID: "2ab84c23-568c-43b0-8b49-597a60b39e2c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:28:47.043802 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.043770 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ab84c23-568c-43b0-8b49-597a60b39e2c-tls-certs\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:47.043802 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.043795 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-home\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:47.043802 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.043804 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm9f6\" (UniqueName: \"kubernetes.io/projected/2ab84c23-568c-43b0-8b49-597a60b39e2c-kube-api-access-hm9f6\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:47.044063 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.043814 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ab84c23-568c-43b0-8b49-597a60b39e2c-dshm\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:28:47.323609 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.323517 2585 generic.go:356] "Generic (PLEG): container finished" podID="2ab84c23-568c-43b0-8b49-597a60b39e2c" containerID="117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1" exitCode=0 Jul 18 01:28:47.323609 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.323565 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" Jul 18 01:28:47.323808 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.323610 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" event={"ID":"2ab84c23-568c-43b0-8b49-597a60b39e2c","Type":"ContainerDied","Data":"117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1"} Jul 18 01:28:47.323808 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.323651 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp" event={"ID":"2ab84c23-568c-43b0-8b49-597a60b39e2c","Type":"ContainerDied","Data":"eaea8a34f1c387ddd9bc04442196cca15f30082dbe2ffd2f0129cba00e19e060"} Jul 18 01:28:47.323808 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.323671 2585 scope.go:117] "RemoveContainer" containerID="117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1" Jul 18 01:28:47.339290 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.339263 2585 scope.go:117] "RemoveContainer" containerID="117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1" Jul 18 01:28:47.339564 ip-10-0-135-148 kubenswrapper[2585]: E0718 01:28:47.339543 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1\": container with ID starting with 117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1 not found: ID does not exist" containerID="117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1" Jul 18 01:28:47.339623 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.339573 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1"} err="failed to get container status \"117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1\": rpc error: code = NotFound desc = could not find container \"117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1\": container with ID starting with 117193b079f2b5ebf6ac42a9841c28401207b51dac5fdb6796468b1ed93216a1 not found: ID does not exist" Jul 18 01:28:47.348868 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.348841 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp"] Jul 18 01:28:47.354185 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.354161 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0a3406/test-model-0a3406-kserve-778bc49dd9-849gp"] Jul 18 01:28:47.456514 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.456479 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ab84c23-568c-43b0-8b49-597a60b39e2c" path="/var/lib/kubelet/pods/2ab84c23-568c-43b0-8b49-597a60b39e2c/volumes" Jul 18 01:28:47.456898 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:47.456885 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aeb9a35-556d-4c12-a58f-5bd543c2939c" path="/var/lib/kubelet/pods/3aeb9a35-556d-4c12-a58f-5bd543c2939c/volumes" Jul 18 01:28:48.850565 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:48.850524 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:52.942057 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:52.942019 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:53.086986 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:53.086949 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:57.508801 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:57.508767 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:57.728555 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:57.728521 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:58.353247 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:58.353205 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:28:58.455176 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:28:58.455135 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:08.246123 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:08.246089 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:12.927711 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:12.927669 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:18.727678 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:18.727640 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:18.831459 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:18.831420 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:23.578447 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:23.578404 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:23.819713 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:23.819680 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:24.143555 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:24.143518 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:24.448410 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:24.448378 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:25.039119 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:25.039092 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:27.373165 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:27.373115 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:27.449356 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:27.449305 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:27.518073 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:27.518040 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:27.934475 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:27.934437 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:28.035461 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:28.035424 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:28.136201 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:28.136162 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:31.866621 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:31.866590 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:32.326258 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:32.326219 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:32.609337 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:32.609241 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:32.719486 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:32.719449 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:35.439869 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:35.439814 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:35.520461 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:35.520408 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:35.601083 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:35.601042 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:36.147799 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:36.147753 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:36.261421 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:36.261386 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:36.361543 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:36.361504 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:39.918690 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:39.918646 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:48.461272 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:48.461229 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:48.565361 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:48.565324 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:57.172932 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:57.172894 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:57.276215 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:57.276179 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:58.345135 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:58.345102 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:58.445601 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:58.445556 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:29:58.601001 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:29:58.600923 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:02.672213 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:02.672178 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:02.907923 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:02.907887 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:08.867451 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:08.867411 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:08.936940 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:08.936905 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:13.606437 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:13.606401 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:13.915564 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:13.915512 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:14.493884 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:14.493849 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:14.595181 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:14.595144 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:24.583189 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:24.583147 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:29.075018 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:29.074984 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:29.591071 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:29.591033 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:30:29.594880 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:29.594857 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:30:29.598331 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:29.598309 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:30:29.601870 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:29.601853 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:30:34.661353 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:34.661313 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:34.761669 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:34.761638 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:39.754783 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:39.754751 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:39.971024 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:39.970991 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:40.309974 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:40.309935 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:40.621181 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:40.621097 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:40.821119 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:40.821082 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:43.519813 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:43.519776 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:43.624915 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:43.624881 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:43.832995 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:43.832889 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:43.921813 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:43.921775 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:43.992263 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:43.992222 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:44.147447 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:44.147423 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:44.254321 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:44.254293 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:44.492319 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:44.492232 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:44.593756 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:44.593723 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:48.351340 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:48.351303 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:48.686729 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:48.686693 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:49.009275 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:49.009174 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:49.318320 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:49.318241 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:51.911134 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:51.911081 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:51.983282 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:51.983243 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:52.056485 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:52.056452 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:52.227930 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:52.227808 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:52.698338 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:52.698305 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:52.800620 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:52.800585 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:30:56.413122 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:30:56.413060 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:31:06.164425 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:06.164374 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:31:06.273058 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:06.273017 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:31:07.730636 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.730602 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss"] Jul 18 01:31:07.731052 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.730945 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aeb9a35-556d-4c12-a58f-5bd543c2939c" containerName="main" Jul 18 01:31:07.731052 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.730957 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aeb9a35-556d-4c12-a58f-5bd543c2939c" containerName="main" Jul 18 01:31:07.731052 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.730972 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ab84c23-568c-43b0-8b49-597a60b39e2c" containerName="main" Jul 18 01:31:07.731052 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.730978 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ab84c23-568c-43b0-8b49-597a60b39e2c" containerName="main" Jul 18 01:31:07.731052 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.731028 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aeb9a35-556d-4c12-a58f-5bd543c2939c" containerName="main" Jul 18 01:31:07.731052 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.731040 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ab84c23-568c-43b0-8b49-597a60b39e2c" containerName="main" Jul 18 01:31:07.733895 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.733877 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.737772 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.737751 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ad1b67\"/\"openshift-service-ca.crt\"" Jul 18 01:31:07.737950 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.737794 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ad1b67\"/\"kube-root-ca.crt\"" Jul 18 01:31:07.737950 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.737845 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ad1b67\"/\"auth-test-model-ad1b67-kserve-self-signed-certs\"" Jul 18 01:31:07.737950 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.737869 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ad1b67\"/\"default-dockercfg-ld4nr\"" Jul 18 01:31:07.745120 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.745094 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss"] Jul 18 01:31:07.760316 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.760287 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:31:07.810655 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.810620 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-dshm\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.810655 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.810654 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-model-cache\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.810988 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.810675 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0013782e-c164-46a4-b253-924e23204d90-tls-certs\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.810988 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.810692 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zswrk\" (UniqueName: \"kubernetes.io/projected/0013782e-c164-46a4-b253-924e23204d90-kube-api-access-zswrk\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.810988 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.810764 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-home\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.810988 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.810800 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-tmp-dir\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.911897 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.911824 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-dshm\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.911897 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.911902 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-model-cache\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.912136 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.911931 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0013782e-c164-46a4-b253-924e23204d90-tls-certs\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.912136 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.911955 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zswrk\" (UniqueName: \"kubernetes.io/projected/0013782e-c164-46a4-b253-924e23204d90-kube-api-access-zswrk\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.912136 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.912021 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-home\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.912136 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.912065 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-tmp-dir\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.912418 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.912390 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-model-cache\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.912538 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.912461 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-tmp-dir\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.912538 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.912481 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-home\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.914323 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.914303 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-dshm\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.914433 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.914415 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0013782e-c164-46a4-b253-924e23204d90-tls-certs\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:07.920041 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:07.920018 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zswrk\" (UniqueName: \"kubernetes.io/projected/0013782e-c164-46a4-b253-924e23204d90-kube-api-access-zswrk\") pod \"auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:08.042651 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:08.042554 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:08.176582 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:08.176555 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss"] Jul 18 01:31:08.181797 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:31:08.181766 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0013782e_c164_46a4_b253_924e23204d90.slice/crio-017fafceb2b86f48ed4a3aacbacd5b695264e9563e839824a9806580d742d241 WatchSource:0}: Error finding container 017fafceb2b86f48ed4a3aacbacd5b695264e9563e839824a9806580d742d241: Status 404 returned error can't find the container with id 017fafceb2b86f48ed4a3aacbacd5b695264e9563e839824a9806580d742d241 Jul 18 01:31:08.956203 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:08.956166 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" event={"ID":"0013782e-c164-46a4-b253-924e23204d90","Type":"ContainerStarted","Data":"d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960"} Jul 18 01:31:08.956592 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:08.956214 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" event={"ID":"0013782e-c164-46a4-b253-924e23204d90","Type":"ContainerStarted","Data":"017fafceb2b86f48ed4a3aacbacd5b695264e9563e839824a9806580d742d241"} Jul 18 01:31:08.973714 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:08.973657 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" podStartSLOduration=1.973642857 podStartE2EDuration="1.973642857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:31:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:31:08.972349203 +0000 UTC m=+3039.994483989" watchObservedRunningTime="2026-07-18 01:31:08.973642857 +0000 UTC m=+3039.995777683" Jul 18 01:31:13.625756 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:13.625722 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:31:13.723627 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:13.723579 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:31:18.043185 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:18.043146 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:18.043789 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:18.043201 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:18.056194 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:18.056163 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:19.015004 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:19.014967 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:31:36.842287 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.842211 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt"] Jul 18 01:31:36.844597 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.844579 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:36.847253 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.847228 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"openshift-service-ca.crt\"" Jul 18 01:31:36.847380 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.847229 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"kube-root-ca.crt\"" Jul 18 01:31:36.848587 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.848570 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"default-dockercfg-tdm25\"" Jul 18 01:31:36.848690 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.848638 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"auth-test-model-0f7fdc-kserve-self-signed-certs\"" Jul 18 01:31:36.856365 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.856344 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt"] Jul 18 01:31:36.887685 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.887650 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:31:36.942859 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.942810 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tmp-dir\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:36.943038 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.942921 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-model-cache\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:36.943038 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.942989 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc8rj\" (UniqueName: \"kubernetes.io/projected/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-kube-api-access-mc8rj\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:36.943038 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.943017 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-dshm\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:36.943152 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.943102 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tls-certs\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:36.943152 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:36.943141 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-home\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.044227 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.044188 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-dshm\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.044425 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.044260 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tls-certs\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.044425 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.044288 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-home\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.044425 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.044313 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tmp-dir\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.044425 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.044386 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-model-cache\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.044650 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.044437 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc8rj\" (UniqueName: \"kubernetes.io/projected/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-kube-api-access-mc8rj\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.044741 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.044720 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-home\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.044803 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.044771 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tmp-dir\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.044857 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.044806 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-model-cache\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.046624 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.046605 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-dshm\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.046924 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.046903 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tls-certs\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.052946 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.052922 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc8rj\" (UniqueName: \"kubernetes.io/projected/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-kube-api-access-mc8rj\") pod \"auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.154008 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.153973 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:37.288068 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:37.287815 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt"] Jul 18 01:31:37.290514 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:31:37.290483 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98c5c08f_e9f5_4ab5_841c_a73e23a4821f.slice/crio-f601e6bfbceb96c5ffda219e25dfaa68461672fc2a466db81e15cebf73802647 WatchSource:0}: Error finding container f601e6bfbceb96c5ffda219e25dfaa68461672fc2a466db81e15cebf73802647: Status 404 returned error can't find the container with id f601e6bfbceb96c5ffda219e25dfaa68461672fc2a466db81e15cebf73802647 Jul 18 01:31:38.092380 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:38.092341 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" event={"ID":"98c5c08f-e9f5-4ab5-841c-a73e23a4821f","Type":"ContainerStarted","Data":"807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd"} Jul 18 01:31:38.092916 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:38.092390 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" event={"ID":"98c5c08f-e9f5-4ab5-841c-a73e23a4821f","Type":"ContainerStarted","Data":"f601e6bfbceb96c5ffda219e25dfaa68461672fc2a466db81e15cebf73802647"} Jul 18 01:31:38.110607 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:38.110555 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" podStartSLOduration=2.110534851 podStartE2EDuration="2.110534851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:31:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:31:38.109304553 +0000 UTC m=+3069.131439338" watchObservedRunningTime="2026-07-18 01:31:38.110534851 +0000 UTC m=+3069.132669635" Jul 18 01:31:47.154097 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:47.154061 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:47.154515 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:47.154289 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:47.166725 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:47.166703 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:31:48.148566 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:31:48.148535 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:32:11.255795 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:32:11.255760 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:32:52.988909 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:32:52.988868 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:32:53.043692 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:32:53.043646 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:33:46.947342 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:33:46.947303 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss"] Jul 18 01:33:46.967886 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:33:46.967821 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:33:47.928467 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:33:47.928431 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt"] Jul 18 01:33:48.285579 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:33:48.285476 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:34:01.969743 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:01.969673 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" podUID="0013782e-c164-46a4-b253-924e23204d90" containerName="main" containerID="cri-o://d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960" gracePeriod=45 Jul 18 01:34:02.216384 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.216358 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:34:02.244330 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.244127 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc"] Jul 18 01:34:02.244624 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.244606 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0013782e-c164-46a4-b253-924e23204d90" containerName="main" Jul 18 01:34:02.244700 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.244627 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0013782e-c164-46a4-b253-924e23204d90" containerName="main" Jul 18 01:34:02.244748 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.244737 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0013782e-c164-46a4-b253-924e23204d90" containerName="main" Jul 18 01:34:02.247764 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.247747 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.250188 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.250169 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ad1b67\"/\"sub-test-model-ad1b67-kserve-self-signed-certs\"" Jul 18 01:34:02.257955 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.257928 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc"] Jul 18 01:34:02.277535 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.277483 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:34:02.279257 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.279238 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-dshm\") pod \"0013782e-c164-46a4-b253-924e23204d90\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " Jul 18 01:34:02.279336 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.279306 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-model-cache\") pod \"0013782e-c164-46a4-b253-924e23204d90\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " Jul 18 01:34:02.279399 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.279342 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0013782e-c164-46a4-b253-924e23204d90-tls-certs\") pod \"0013782e-c164-46a4-b253-924e23204d90\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " Jul 18 01:34:02.279399 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.279363 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-home\") pod \"0013782e-c164-46a4-b253-924e23204d90\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " Jul 18 01:34:02.279399 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.279380 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-tmp-dir\") pod \"0013782e-c164-46a4-b253-924e23204d90\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " Jul 18 01:34:02.279570 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.279408 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zswrk\" (UniqueName: \"kubernetes.io/projected/0013782e-c164-46a4-b253-924e23204d90-kube-api-access-zswrk\") pod \"0013782e-c164-46a4-b253-924e23204d90\" (UID: \"0013782e-c164-46a4-b253-924e23204d90\") " Jul 18 01:34:02.279623 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.279571 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-model-cache" (OuterVolumeSpecName: "model-cache") pod "0013782e-c164-46a4-b253-924e23204d90" (UID: "0013782e-c164-46a4-b253-924e23204d90"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:34:02.279748 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.279731 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-model-cache\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:02.279804 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.279713 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0013782e-c164-46a4-b253-924e23204d90" (UID: "0013782e-c164-46a4-b253-924e23204d90"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:34:02.279917 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.279822 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-home" (OuterVolumeSpecName: "home") pod "0013782e-c164-46a4-b253-924e23204d90" (UID: "0013782e-c164-46a4-b253-924e23204d90"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:34:02.281815 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.281795 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0013782e-c164-46a4-b253-924e23204d90-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0013782e-c164-46a4-b253-924e23204d90" (UID: "0013782e-c164-46a4-b253-924e23204d90"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:34:02.281940 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.281911 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-dshm" (OuterVolumeSpecName: "dshm") pod "0013782e-c164-46a4-b253-924e23204d90" (UID: "0013782e-c164-46a4-b253-924e23204d90"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:34:02.282069 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.281949 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0013782e-c164-46a4-b253-924e23204d90-kube-api-access-zswrk" (OuterVolumeSpecName: "kube-api-access-zswrk") pod "0013782e-c164-46a4-b253-924e23204d90" (UID: "0013782e-c164-46a4-b253-924e23204d90"). InnerVolumeSpecName "kube-api-access-zswrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:34:02.380535 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.380495 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-model-cache\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.380535 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.380535 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-home\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.380746 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.380563 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tmp-dir\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.380746 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.380593 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9v2d\" (UniqueName: \"kubernetes.io/projected/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-kube-api-access-p9v2d\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.380746 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.380626 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-dshm\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.380746 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.380710 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tls-certs\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.380919 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.380780 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0013782e-c164-46a4-b253-924e23204d90-tls-certs\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:02.380919 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.380795 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-home\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:02.380919 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.380807 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-tmp-dir\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:02.380919 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.380816 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zswrk\" (UniqueName: \"kubernetes.io/projected/0013782e-c164-46a4-b253-924e23204d90-kube-api-access-zswrk\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:02.380919 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.380825 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0013782e-c164-46a4-b253-924e23204d90-dshm\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:02.481984 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.481933 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tls-certs\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.482131 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.482017 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-model-cache\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.482131 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.482042 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-home\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.482131 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.482062 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tmp-dir\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.482131 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.482085 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9v2d\" (UniqueName: \"kubernetes.io/projected/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-kube-api-access-p9v2d\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.482131 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.482113 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-dshm\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.482488 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.482461 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-home\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.482581 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.482519 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-model-cache\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.482581 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.482547 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tmp-dir\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.484348 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.484324 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-dshm\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.484617 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.484601 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tls-certs\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.490978 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.490953 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9v2d\" (UniqueName: \"kubernetes.io/projected/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-kube-api-access-p9v2d\") pod \"sub-test-model-ad1b67-kserve-76569476b5-mtnhc\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.560059 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.559967 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:02.695135 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.695104 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc"] Jul 18 01:34:02.697171 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:34:02.697139 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3e77dae_ff52_44b7_9b0c_69d26e95ec79.slice/crio-8fee1c0ae819a1bbedac95eafd9259fa678c723e3300504fcb3b757c5e10ead1 WatchSource:0}: Error finding container 8fee1c0ae819a1bbedac95eafd9259fa678c723e3300504fcb3b757c5e10ead1: Status 404 returned error can't find the container with id 8fee1c0ae819a1bbedac95eafd9259fa678c723e3300504fcb3b757c5e10ead1 Jul 18 01:34:02.744290 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.744264 2585 generic.go:356] "Generic (PLEG): container finished" podID="0013782e-c164-46a4-b253-924e23204d90" containerID="d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960" exitCode=0 Jul 18 01:34:02.744423 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.744310 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" Jul 18 01:34:02.744423 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.744347 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" event={"ID":"0013782e-c164-46a4-b253-924e23204d90","Type":"ContainerDied","Data":"d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960"} Jul 18 01:34:02.744423 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.744397 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss" event={"ID":"0013782e-c164-46a4-b253-924e23204d90","Type":"ContainerDied","Data":"017fafceb2b86f48ed4a3aacbacd5b695264e9563e839824a9806580d742d241"} Jul 18 01:34:02.744423 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.744418 2585 scope.go:117] "RemoveContainer" containerID="d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960" Jul 18 01:34:02.745819 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.745792 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" event={"ID":"a3e77dae-ff52-44b7-9b0c-69d26e95ec79","Type":"ContainerStarted","Data":"8fee1c0ae819a1bbedac95eafd9259fa678c723e3300504fcb3b757c5e10ead1"} Jul 18 01:34:02.753081 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.753054 2585 scope.go:117] "RemoveContainer" containerID="d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960" Jul 18 01:34:02.753351 ip-10-0-135-148 kubenswrapper[2585]: E0718 01:34:02.753333 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960\": container with ID starting with d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960 not found: ID does not exist" containerID="d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960" Jul 18 01:34:02.753418 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.753361 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960"} err="failed to get container status \"d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960\": rpc error: code = NotFound desc = could not find container \"d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960\": container with ID starting with d26cf665652094cdd9d8f40f5cf647467e89b05c961ee028e94bf0949fd59960 not found: ID does not exist" Jul 18 01:34:02.768876 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.768815 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss"] Jul 18 01:34:02.772946 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.772918 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/auth-test-model-ad1b67-kserve-54b45cdd6c-5ctss"] Jul 18 01:34:02.950438 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:02.950384 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" podUID="98c5c08f-e9f5-4ab5-841c-a73e23a4821f" containerName="main" containerID="cri-o://807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd" gracePeriod=45 Jul 18 01:34:03.202624 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.202550 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:34:03.290104 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.290062 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mc8rj\" (UniqueName: \"kubernetes.io/projected/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-kube-api-access-mc8rj\") pod \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " Jul 18 01:34:03.290287 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.290129 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tls-certs\") pod \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " Jul 18 01:34:03.290287 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.290168 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tmp-dir\") pod \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " Jul 18 01:34:03.290287 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.290242 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-home\") pod \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " Jul 18 01:34:03.290287 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.290282 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-dshm\") pod \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " Jul 18 01:34:03.290498 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.290351 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-model-cache\") pod \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\" (UID: \"98c5c08f-e9f5-4ab5-841c-a73e23a4821f\") " Jul 18 01:34:03.290498 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.290458 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98c5c08f-e9f5-4ab5-841c-a73e23a4821f" (UID: "98c5c08f-e9f5-4ab5-841c-a73e23a4821f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:34:03.290590 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.290529 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-home" (OuterVolumeSpecName: "home") pod "98c5c08f-e9f5-4ab5-841c-a73e23a4821f" (UID: "98c5c08f-e9f5-4ab5-841c-a73e23a4821f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:34:03.290667 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.290637 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-model-cache" (OuterVolumeSpecName: "model-cache") pod "98c5c08f-e9f5-4ab5-841c-a73e23a4821f" (UID: "98c5c08f-e9f5-4ab5-841c-a73e23a4821f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:34:03.290723 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.290706 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tmp-dir\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:03.290765 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.290729 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-home\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:03.292604 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.292578 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-dshm" (OuterVolumeSpecName: "dshm") pod "98c5c08f-e9f5-4ab5-841c-a73e23a4821f" (UID: "98c5c08f-e9f5-4ab5-841c-a73e23a4821f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:34:03.292604 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.292595 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98c5c08f-e9f5-4ab5-841c-a73e23a4821f" (UID: "98c5c08f-e9f5-4ab5-841c-a73e23a4821f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:34:03.292759 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.292598 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-kube-api-access-mc8rj" (OuterVolumeSpecName: "kube-api-access-mc8rj") pod "98c5c08f-e9f5-4ab5-841c-a73e23a4821f" (UID: "98c5c08f-e9f5-4ab5-841c-a73e23a4821f"). InnerVolumeSpecName "kube-api-access-mc8rj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:34:03.391965 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.391929 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-dshm\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:03.391965 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.391960 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-model-cache\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:03.391965 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.391969 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mc8rj\" (UniqueName: \"kubernetes.io/projected/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-kube-api-access-mc8rj\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:03.392182 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.391980 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98c5c08f-e9f5-4ab5-841c-a73e23a4821f-tls-certs\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:34:03.456238 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.456160 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0013782e-c164-46a4-b253-924e23204d90" path="/var/lib/kubelet/pods/0013782e-c164-46a4-b253-924e23204d90/volumes" Jul 18 01:34:03.752530 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.752444 2585 generic.go:356] "Generic (PLEG): container finished" podID="98c5c08f-e9f5-4ab5-841c-a73e23a4821f" containerID="807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd" exitCode=0 Jul 18 01:34:03.752530 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.752490 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" Jul 18 01:34:03.752773 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.752531 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" event={"ID":"98c5c08f-e9f5-4ab5-841c-a73e23a4821f","Type":"ContainerDied","Data":"807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd"} Jul 18 01:34:03.752773 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.752575 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt" event={"ID":"98c5c08f-e9f5-4ab5-841c-a73e23a4821f","Type":"ContainerDied","Data":"f601e6bfbceb96c5ffda219e25dfaa68461672fc2a466db81e15cebf73802647"} Jul 18 01:34:03.752773 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.752599 2585 scope.go:117] "RemoveContainer" containerID="807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd" Jul 18 01:34:03.754184 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.754157 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" event={"ID":"a3e77dae-ff52-44b7-9b0c-69d26e95ec79","Type":"ContainerStarted","Data":"acec8fc8a6c1d830350b5e9063fbbd796fe2c8c29ef720690a0f14caa07c45cc"} Jul 18 01:34:03.761369 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.761348 2585 scope.go:117] "RemoveContainer" containerID="807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd" Jul 18 01:34:03.761653 ip-10-0-135-148 kubenswrapper[2585]: E0718 01:34:03.761623 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd\": container with ID starting with 807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd not found: ID does not exist" containerID="807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd" Jul 18 01:34:03.761760 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.761660 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd"} err="failed to get container status \"807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd\": rpc error: code = NotFound desc = could not find container \"807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd\": container with ID starting with 807a18c08fe78b69d1b0d9c544471ebed5c3dbd2e0486d4b01470c818febb6cd not found: ID does not exist" Jul 18 01:34:03.770621 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.770597 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt"] Jul 18 01:34:03.774968 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.774936 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/auth-test-model-0f7fdc-kserve-64fd749bf9-79vlt"] Jul 18 01:34:03.789270 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:03.789220 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" podStartSLOduration=1.789202565 podStartE2EDuration="1.789202565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:34:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:34:03.788294752 +0000 UTC m=+3214.810429539" watchObservedRunningTime="2026-07-18 01:34:03.789202565 +0000 UTC m=+3214.811337349" Jul 18 01:34:05.457082 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:05.457050 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98c5c08f-e9f5-4ab5-841c-a73e23a4821f" path="/var/lib/kubelet/pods/98c5c08f-e9f5-4ab5-841c-a73e23a4821f/volumes" Jul 18 01:34:12.560990 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:12.560954 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:12.561402 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:12.561006 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:12.573758 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:12.573730 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:12.811225 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:12.811141 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:34:28.340925 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.340822 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z"] Jul 18 01:34:28.341411 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.341245 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c5c08f-e9f5-4ab5-841c-a73e23a4821f" containerName="main" Jul 18 01:34:28.341411 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.341260 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c5c08f-e9f5-4ab5-841c-a73e23a4821f" containerName="main" Jul 18 01:34:28.341411 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.341320 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c5c08f-e9f5-4ab5-841c-a73e23a4821f" containerName="main" Jul 18 01:34:28.344299 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.344278 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.347928 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.347909 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"kube-root-ca.crt\"" Jul 18 01:34:28.348164 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.348142 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"default-dockercfg-tdm25\"" Jul 18 01:34:28.348164 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.348165 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"sub-test-model-0f7fdc-kserve-self-signed-certs\"" Jul 18 01:34:28.348164 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.348170 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"openshift-service-ca.crt\"" Jul 18 01:34:28.352769 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.352750 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z"] Jul 18 01:34:28.370096 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.370063 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:34:28.413315 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.413282 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-model-cache\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.413315 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.413317 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7fba57f-887f-4fe8-af13-cf4110942f9c-tls-certs\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.413492 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.413360 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-home\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.413492 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.413424 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-tmp-dir\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.413492 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.413474 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldqrr\" (UniqueName: \"kubernetes.io/projected/c7fba57f-887f-4fe8-af13-cf4110942f9c-kube-api-access-ldqrr\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.413611 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.413498 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-dshm\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.514270 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.514233 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-home\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.514461 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.514301 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-tmp-dir\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.514461 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.514373 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldqrr\" (UniqueName: \"kubernetes.io/projected/c7fba57f-887f-4fe8-af13-cf4110942f9c-kube-api-access-ldqrr\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.514461 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.514405 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-dshm\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.514461 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.514432 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-model-cache\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.514710 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.514580 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7fba57f-887f-4fe8-af13-cf4110942f9c-tls-certs\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.514710 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.514682 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-home\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.514818 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.514738 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-tmp-dir\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.514818 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.514762 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-model-cache\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.516722 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.516693 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-dshm\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.517030 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.517012 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7fba57f-887f-4fe8-af13-cf4110942f9c-tls-certs\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.523135 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.523107 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldqrr\" (UniqueName: \"kubernetes.io/projected/c7fba57f-887f-4fe8-af13-cf4110942f9c-kube-api-access-ldqrr\") pod \"sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.654479 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.654442 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:28.794012 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.793789 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z"] Jul 18 01:34:28.796338 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:34:28.796310 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7fba57f_887f_4fe8_af13_cf4110942f9c.slice/crio-2c8d1fd90eebadcdd8d4b7c5f98488247c3f70a89e321656cf1d3a854463fb90 WatchSource:0}: Error finding container 2c8d1fd90eebadcdd8d4b7c5f98488247c3f70a89e321656cf1d3a854463fb90: Status 404 returned error can't find the container with id 2c8d1fd90eebadcdd8d4b7c5f98488247c3f70a89e321656cf1d3a854463fb90 Jul 18 01:34:28.870325 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.870288 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" event={"ID":"c7fba57f-887f-4fe8-af13-cf4110942f9c","Type":"ContainerStarted","Data":"9a8df79d24fb0f8e479aff0bf2cfb78c928b6b4cc268afc137c5bcde66a7f749"} Jul 18 01:34:28.870488 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.870340 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" event={"ID":"c7fba57f-887f-4fe8-af13-cf4110942f9c","Type":"ContainerStarted","Data":"2c8d1fd90eebadcdd8d4b7c5f98488247c3f70a89e321656cf1d3a854463fb90"} Jul 18 01:34:28.886555 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:28.886502 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" podStartSLOduration=0.886486862 podStartE2EDuration="886.486862ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:34:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:34:28.885723321 +0000 UTC m=+3239.907858140" watchObservedRunningTime="2026-07-18 01:34:28.886486862 +0000 UTC m=+3239.908621713" Jul 18 01:34:34.340365 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:34.340333 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:34:34.440947 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:34.440913 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:34:38.654882 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:38.654799 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:38.654882 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:38.654889 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:38.667648 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:38.667619 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:38.927092 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:38.927002 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:34:53.493308 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:53.493267 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:34:53.594367 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:34:53.594334 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:35:19.917598 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:19.917559 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc"] Jul 18 01:35:20.689320 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:20.689284 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:35:20.938405 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:20.938375 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z"] Jul 18 01:35:20.957274 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:20.957196 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:35:29.617879 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:29.617815 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:35:29.622044 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:29.622019 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:35:29.625804 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:29.625779 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:35:29.629936 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:29.629914 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c6t5d_1c50bca7-78a9-43b3-87a3-74812b70f11b/ovn-acl-logging/0.log" Jul 18 01:35:34.937925 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:34.937875 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" podUID="a3e77dae-ff52-44b7-9b0c-69d26e95ec79" containerName="main" containerID="cri-o://acec8fc8a6c1d830350b5e9063fbbd796fe2c8c29ef720690a0f14caa07c45cc" gracePeriod=45 Jul 18 01:35:35.176018 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.175935 2585 generic.go:356] "Generic (PLEG): container finished" podID="a3e77dae-ff52-44b7-9b0c-69d26e95ec79" containerID="acec8fc8a6c1d830350b5e9063fbbd796fe2c8c29ef720690a0f14caa07c45cc" exitCode=0 Jul 18 01:35:35.176132 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.176013 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" event={"ID":"a3e77dae-ff52-44b7-9b0c-69d26e95ec79","Type":"ContainerDied","Data":"acec8fc8a6c1d830350b5e9063fbbd796fe2c8c29ef720690a0f14caa07c45cc"} Jul 18 01:35:35.176132 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.176063 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" event={"ID":"a3e77dae-ff52-44b7-9b0c-69d26e95ec79","Type":"ContainerDied","Data":"8fee1c0ae819a1bbedac95eafd9259fa678c723e3300504fcb3b757c5e10ead1"} Jul 18 01:35:35.176132 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.176075 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8fee1c0ae819a1bbedac95eafd9259fa678c723e3300504fcb3b757c5e10ead1" Jul 18 01:35:35.190979 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.190921 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:35:35.338752 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.338699 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-dshm\") pod \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " Jul 18 01:35:35.338971 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.338771 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9v2d\" (UniqueName: \"kubernetes.io/projected/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-kube-api-access-p9v2d\") pod \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " Jul 18 01:35:35.338971 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.338793 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-model-cache\") pod \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " Jul 18 01:35:35.338971 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.338888 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tmp-dir\") pod \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " Jul 18 01:35:35.339127 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.338996 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-home\") pod \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " Jul 18 01:35:35.339127 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.339031 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tls-certs\") pod \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\" (UID: \"a3e77dae-ff52-44b7-9b0c-69d26e95ec79\") " Jul 18 01:35:35.339236 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.339139 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a3e77dae-ff52-44b7-9b0c-69d26e95ec79" (UID: "a3e77dae-ff52-44b7-9b0c-69d26e95ec79"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:35:35.339236 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.339171 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-model-cache" (OuterVolumeSpecName: "model-cache") pod "a3e77dae-ff52-44b7-9b0c-69d26e95ec79" (UID: "a3e77dae-ff52-44b7-9b0c-69d26e95ec79"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:35:35.339349 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.339295 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-home" (OuterVolumeSpecName: "home") pod "a3e77dae-ff52-44b7-9b0c-69d26e95ec79" (UID: "a3e77dae-ff52-44b7-9b0c-69d26e95ec79"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:35:35.339403 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.339385 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tmp-dir\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:35.339457 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.339404 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-home\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:35.339457 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.339419 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-model-cache\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:35.341047 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.341019 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-dshm" (OuterVolumeSpecName: "dshm") pod "a3e77dae-ff52-44b7-9b0c-69d26e95ec79" (UID: "a3e77dae-ff52-44b7-9b0c-69d26e95ec79"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:35:35.341047 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.341030 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a3e77dae-ff52-44b7-9b0c-69d26e95ec79" (UID: "a3e77dae-ff52-44b7-9b0c-69d26e95ec79"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:35:35.341183 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.341080 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-kube-api-access-p9v2d" (OuterVolumeSpecName: "kube-api-access-p9v2d") pod "a3e77dae-ff52-44b7-9b0c-69d26e95ec79" (UID: "a3e77dae-ff52-44b7-9b0c-69d26e95ec79"). InnerVolumeSpecName "kube-api-access-p9v2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:35:35.440412 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.440369 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-tls-certs\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:35.440412 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.440403 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-dshm\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:35.440412 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.440412 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9v2d\" (UniqueName: \"kubernetes.io/projected/a3e77dae-ff52-44b7-9b0c-69d26e95ec79-kube-api-access-p9v2d\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:35.958677 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:35.958635 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" podUID="c7fba57f-887f-4fe8-af13-cf4110942f9c" containerName="main" containerID="cri-o://9a8df79d24fb0f8e479aff0bf2cfb78c928b6b4cc268afc137c5bcde66a7f749" gracePeriod=45 Jul 18 01:35:36.182325 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.182291 2585 generic.go:356] "Generic (PLEG): container finished" podID="c7fba57f-887f-4fe8-af13-cf4110942f9c" containerID="9a8df79d24fb0f8e479aff0bf2cfb78c928b6b4cc268afc137c5bcde66a7f749" exitCode=0 Jul 18 01:35:36.182507 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.182363 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" event={"ID":"c7fba57f-887f-4fe8-af13-cf4110942f9c","Type":"ContainerDied","Data":"9a8df79d24fb0f8e479aff0bf2cfb78c928b6b4cc268afc137c5bcde66a7f749"} Jul 18 01:35:36.182507 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.182458 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc" Jul 18 01:35:36.199191 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.199159 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc"] Jul 18 01:35:36.201559 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.201536 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/sub-test-model-ad1b67-kserve-76569476b5-mtnhc"] Jul 18 01:35:36.207742 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.207723 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:35:36.348897 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.348768 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-dshm\") pod \"c7fba57f-887f-4fe8-af13-cf4110942f9c\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " Jul 18 01:35:36.348897 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.348818 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7fba57f-887f-4fe8-af13-cf4110942f9c-tls-certs\") pod \"c7fba57f-887f-4fe8-af13-cf4110942f9c\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " Jul 18 01:35:36.348897 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.348887 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldqrr\" (UniqueName: \"kubernetes.io/projected/c7fba57f-887f-4fe8-af13-cf4110942f9c-kube-api-access-ldqrr\") pod \"c7fba57f-887f-4fe8-af13-cf4110942f9c\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " Jul 18 01:35:36.349185 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.348949 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-model-cache\") pod \"c7fba57f-887f-4fe8-af13-cf4110942f9c\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " Jul 18 01:35:36.349185 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.348969 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-home\") pod \"c7fba57f-887f-4fe8-af13-cf4110942f9c\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " Jul 18 01:35:36.349185 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.349035 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-tmp-dir\") pod \"c7fba57f-887f-4fe8-af13-cf4110942f9c\" (UID: \"c7fba57f-887f-4fe8-af13-cf4110942f9c\") " Jul 18 01:35:36.349331 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.349264 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-model-cache" (OuterVolumeSpecName: "model-cache") pod "c7fba57f-887f-4fe8-af13-cf4110942f9c" (UID: "c7fba57f-887f-4fe8-af13-cf4110942f9c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:35:36.349331 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.349279 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-home" (OuterVolumeSpecName: "home") pod "c7fba57f-887f-4fe8-af13-cf4110942f9c" (UID: "c7fba57f-887f-4fe8-af13-cf4110942f9c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:35:36.349403 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.349364 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c7fba57f-887f-4fe8-af13-cf4110942f9c" (UID: "c7fba57f-887f-4fe8-af13-cf4110942f9c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:35:36.351088 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.351054 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-dshm" (OuterVolumeSpecName: "dshm") pod "c7fba57f-887f-4fe8-af13-cf4110942f9c" (UID: "c7fba57f-887f-4fe8-af13-cf4110942f9c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:35:36.351088 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.351059 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7fba57f-887f-4fe8-af13-cf4110942f9c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c7fba57f-887f-4fe8-af13-cf4110942f9c" (UID: "c7fba57f-887f-4fe8-af13-cf4110942f9c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:35:36.351249 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.351109 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7fba57f-887f-4fe8-af13-cf4110942f9c-kube-api-access-ldqrr" (OuterVolumeSpecName: "kube-api-access-ldqrr") pod "c7fba57f-887f-4fe8-af13-cf4110942f9c" (UID: "c7fba57f-887f-4fe8-af13-cf4110942f9c"). InnerVolumeSpecName "kube-api-access-ldqrr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:35:36.450577 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.450537 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-model-cache\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:36.450577 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.450566 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-home\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:36.450577 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.450577 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-tmp-dir\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:36.450577 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.450584 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7fba57f-887f-4fe8-af13-cf4110942f9c-dshm\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:36.450894 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.450592 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7fba57f-887f-4fe8-af13-cf4110942f9c-tls-certs\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:36.450894 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:36.450600 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldqrr\" (UniqueName: \"kubernetes.io/projected/c7fba57f-887f-4fe8-af13-cf4110942f9c-kube-api-access-ldqrr\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:35:37.188542 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:37.188512 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" Jul 18 01:35:37.188542 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:37.188526 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z" event={"ID":"c7fba57f-887f-4fe8-af13-cf4110942f9c","Type":"ContainerDied","Data":"2c8d1fd90eebadcdd8d4b7c5f98488247c3f70a89e321656cf1d3a854463fb90"} Jul 18 01:35:37.189059 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:37.188582 2585 scope.go:117] "RemoveContainer" containerID="9a8df79d24fb0f8e479aff0bf2cfb78c928b6b4cc268afc137c5bcde66a7f749" Jul 18 01:35:37.209451 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:37.209419 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z"] Jul 18 01:35:37.215284 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:37.215255 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/sub-test-model-0f7fdc-kserve-f7fc9c59f-b7z2z"] Jul 18 01:35:37.457485 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:37.457403 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3e77dae-ff52-44b7-9b0c-69d26e95ec79" path="/var/lib/kubelet/pods/a3e77dae-ff52-44b7-9b0c-69d26e95ec79/volumes" Jul 18 01:35:37.457818 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:37.457802 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7fba57f-887f-4fe8-af13-cf4110942f9c" path="/var/lib/kubelet/pods/c7fba57f-887f-4fe8-af13-cf4110942f9c/volumes" Jul 18 01:35:39.940357 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.940315 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5"] Jul 18 01:35:39.940753 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.940640 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3e77dae-ff52-44b7-9b0c-69d26e95ec79" containerName="main" Jul 18 01:35:39.940753 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.940650 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3e77dae-ff52-44b7-9b0c-69d26e95ec79" containerName="main" Jul 18 01:35:39.940753 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.940671 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7fba57f-887f-4fe8-af13-cf4110942f9c" containerName="main" Jul 18 01:35:39.940753 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.940676 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7fba57f-887f-4fe8-af13-cf4110942f9c" containerName="main" Jul 18 01:35:39.940753 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.940731 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7fba57f-887f-4fe8-af13-cf4110942f9c" containerName="main" Jul 18 01:35:39.940753 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.940742 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3e77dae-ff52-44b7-9b0c-69d26e95ec79" containerName="main" Jul 18 01:35:39.944666 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.944647 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:39.947177 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.947154 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ad1b67\"/\"kube-root-ca.crt\"" Jul 18 01:35:39.947295 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.947279 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ad1b67\"/\"default-dockercfg-ld4nr\"" Jul 18 01:35:39.948270 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.948252 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ad1b67\"/\"openshift-service-ca.crt\"" Jul 18 01:35:39.948525 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.948508 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ad1b67\"/\"rate-test-model-ad1b67-kserve-self-signed-certs\"" Jul 18 01:35:39.953006 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:39.952981 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5"] Jul 18 01:35:40.041403 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.041364 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:35:40.083690 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.083654 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26fhm\" (UniqueName: \"kubernetes.io/projected/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-kube-api-access-26fhm\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.083896 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.083731 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-model-cache\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.083896 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.083759 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tls-certs\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.083896 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.083780 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-dshm\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.083896 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.083821 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-home\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.083896 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.083876 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tmp-dir\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.184767 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.184731 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26fhm\" (UniqueName: \"kubernetes.io/projected/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-kube-api-access-26fhm\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.184957 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.184808 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-model-cache\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.184957 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.184864 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tls-certs\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.184957 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.184896 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-dshm\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.184957 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.184926 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-home\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.184957 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.184948 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tmp-dir\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.185287 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.185262 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-model-cache\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.185349 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.185287 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tmp-dir\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.185349 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.185336 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-home\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.187265 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.187240 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-dshm\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.187529 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.187511 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tls-certs\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.192581 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.192509 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26fhm\" (UniqueName: \"kubernetes.io/projected/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-kube-api-access-26fhm\") pod \"rate-test-model-ad1b67-kserve-84897c7647-t6mh5\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.254899 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.254859 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:40.395455 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:40.395421 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5"] Jul 18 01:35:40.398629 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:35:40.398591 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84bde4be_b5b2_454b_a6fd_ad3f3995d61f.slice/crio-9727f146230d1605617d2c36b87f001de32563edc07a551ee5faf73cfc4a9b79 WatchSource:0}: Error finding container 9727f146230d1605617d2c36b87f001de32563edc07a551ee5faf73cfc4a9b79: Status 404 returned error can't find the container with id 9727f146230d1605617d2c36b87f001de32563edc07a551ee5faf73cfc4a9b79 Jul 18 01:35:41.211962 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:41.211928 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" event={"ID":"84bde4be-b5b2-454b-a6fd-ad3f3995d61f","Type":"ContainerStarted","Data":"eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6"} Jul 18 01:35:41.212378 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:41.211975 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" event={"ID":"84bde4be-b5b2-454b-a6fd-ad3f3995d61f","Type":"ContainerStarted","Data":"9727f146230d1605617d2c36b87f001de32563edc07a551ee5faf73cfc4a9b79"} Jul 18 01:35:41.228073 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:41.228012 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" podStartSLOduration=2.227997286 podStartE2EDuration="2.227997286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:35:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:35:41.226604051 +0000 UTC m=+3312.248738837" watchObservedRunningTime="2026-07-18 01:35:41.227997286 +0000 UTC m=+3312.250132069" Jul 18 01:35:44.348394 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:44.348352 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:35:44.452470 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:44.452435 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:35:50.255517 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:50.255478 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:50.255982 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:50.255536 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:50.268183 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:50.268153 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:51.270978 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:51.270947 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:35:54.723110 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.723076 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb"] Jul 18 01:35:54.726204 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.726179 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.728634 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.728611 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"openshift-service-ca.crt\"" Jul 18 01:35:54.728758 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.728647 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"kube-root-ca.crt\"" Jul 18 01:35:54.729772 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.729754 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"default-dockercfg-tdm25\"" Jul 18 01:35:54.729911 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.729773 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0f7fdc\"/\"rate-test-model-0f7fdc-kserve-self-signed-certs\"" Jul 18 01:35:54.735062 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.735043 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb"] Jul 18 01:35:54.764773 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.764745 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:35:54.803177 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.803147 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tls-certs\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.803332 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.803187 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-home\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.803332 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.803238 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtxn5\" (UniqueName: \"kubernetes.io/projected/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-kube-api-access-dtxn5\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.803332 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.803281 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tmp-dir\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.803332 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.803306 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-model-cache\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.803480 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.803359 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-dshm\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.904516 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.904479 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tls-certs\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.904698 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.904527 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-home\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.904698 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.904642 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtxn5\" (UniqueName: \"kubernetes.io/projected/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-kube-api-access-dtxn5\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.904698 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.904689 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tmp-dir\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.904888 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.904812 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-model-cache\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.904950 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.904911 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-home\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.904950 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.904940 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-dshm\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.905053 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.904999 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tmp-dir\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.905142 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.905126 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-model-cache\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.906881 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.906857 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-dshm\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.907034 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.907017 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tls-certs\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:54.912038 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:54.912010 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtxn5\" (UniqueName: \"kubernetes.io/projected/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-kube-api-access-dtxn5\") pod \"rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:55.039586 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:55.039493 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:35:55.173402 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:55.173354 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb"] Jul 18 01:35:55.175591 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:35:55.175559 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod616cfb97_9357_46a7_9f85_1e3ed80f6e9a.slice/crio-16b36696c214bc6cc62ffba15d4796b3ea43902f0b8ffab53779395439a5e6bd WatchSource:0}: Error finding container 16b36696c214bc6cc62ffba15d4796b3ea43902f0b8ffab53779395439a5e6bd: Status 404 returned error can't find the container with id 16b36696c214bc6cc62ffba15d4796b3ea43902f0b8ffab53779395439a5e6bd Jul 18 01:35:55.278514 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:55.278476 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" event={"ID":"616cfb97-9357-46a7-9f85-1e3ed80f6e9a","Type":"ContainerStarted","Data":"a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149"} Jul 18 01:35:55.278721 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:55.278526 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" event={"ID":"616cfb97-9357-46a7-9f85-1e3ed80f6e9a","Type":"ContainerStarted","Data":"16b36696c214bc6cc62ffba15d4796b3ea43902f0b8ffab53779395439a5e6bd"} Jul 18 01:35:55.296112 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:35:55.296003 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" podStartSLOduration=1.295984502 podStartE2EDuration="1.295984502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:35:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:35:55.294367976 +0000 UTC m=+3326.316502773" watchObservedRunningTime="2026-07-18 01:35:55.295984502 +0000 UTC m=+3326.318119286" Jul 18 01:36:05.039854 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:05.039793 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:36:05.040395 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:05.039878 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:36:05.052908 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:05.052872 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:36:05.337434 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:05.337343 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:36:06.924905 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:06.924866 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:36:07.028388 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:07.028355 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:36:14.370575 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:14.370539 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:36:14.471844 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:14.471783 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:36:43.924551 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:43.924514 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5"] Jul 18 01:36:43.947257 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:43.947224 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:36:44.943624 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:44.943590 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb"] Jul 18 01:36:44.976954 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:44.976901 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:36:58.944758 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:58.944718 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" podUID="84bde4be-b5b2-454b-a6fd-ad3f3995d61f" containerName="main" containerID="cri-o://eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6" gracePeriod=45 Jul 18 01:36:59.204379 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.204319 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:36:59.287927 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.287889 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-model-cache\") pod \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " Jul 18 01:36:59.288113 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.287944 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tls-certs\") pod \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " Jul 18 01:36:59.288113 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.288013 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26fhm\" (UniqueName: \"kubernetes.io/projected/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-kube-api-access-26fhm\") pod \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " Jul 18 01:36:59.288113 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.288050 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tmp-dir\") pod \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " Jul 18 01:36:59.288113 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.288069 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-dshm\") pod \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " Jul 18 01:36:59.288113 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.288085 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-home\") pod \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\" (UID: \"84bde4be-b5b2-454b-a6fd-ad3f3995d61f\") " Jul 18 01:36:59.288397 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.288232 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-model-cache" (OuterVolumeSpecName: "model-cache") pod "84bde4be-b5b2-454b-a6fd-ad3f3995d61f" (UID: "84bde4be-b5b2-454b-a6fd-ad3f3995d61f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:36:59.288397 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.288355 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "84bde4be-b5b2-454b-a6fd-ad3f3995d61f" (UID: "84bde4be-b5b2-454b-a6fd-ad3f3995d61f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:36:59.288547 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.288366 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-model-cache\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:36:59.288547 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.288452 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-home" (OuterVolumeSpecName: "home") pod "84bde4be-b5b2-454b-a6fd-ad3f3995d61f" (UID: "84bde4be-b5b2-454b-a6fd-ad3f3995d61f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:36:59.290344 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.290319 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-dshm" (OuterVolumeSpecName: "dshm") pod "84bde4be-b5b2-454b-a6fd-ad3f3995d61f" (UID: "84bde4be-b5b2-454b-a6fd-ad3f3995d61f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:36:59.290344 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.290332 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84bde4be-b5b2-454b-a6fd-ad3f3995d61f" (UID: "84bde4be-b5b2-454b-a6fd-ad3f3995d61f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:36:59.290490 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.290446 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-kube-api-access-26fhm" (OuterVolumeSpecName: "kube-api-access-26fhm") pod "84bde4be-b5b2-454b-a6fd-ad3f3995d61f" (UID: "84bde4be-b5b2-454b-a6fd-ad3f3995d61f"). InnerVolumeSpecName "kube-api-access-26fhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:36:59.389008 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.388967 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tmp-dir\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:36:59.389008 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.388999 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-dshm\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:36:59.389008 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.389011 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-home\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:36:59.389250 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.389022 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-tls-certs\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:36:59.389250 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.389034 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26fhm\" (UniqueName: \"kubernetes.io/projected/84bde4be-b5b2-454b-a6fd-ad3f3995d61f-kube-api-access-26fhm\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:36:59.539786 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.539749 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq"] Jul 18 01:36:59.540118 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.540104 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84bde4be-b5b2-454b-a6fd-ad3f3995d61f" containerName="main" Jul 18 01:36:59.540160 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.540119 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="84bde4be-b5b2-454b-a6fd-ad3f3995d61f" containerName="main" Jul 18 01:36:59.540205 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.540184 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="84bde4be-b5b2-454b-a6fd-ad3f3995d61f" containerName="main" Jul 18 01:36:59.543217 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.543192 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" Jul 18 01:36:59.545814 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.545796 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmz4m\"" Jul 18 01:36:59.545814 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.545800 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 01:36:59.550344 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.550319 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq"] Jul 18 01:36:59.578928 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.578877 2585 generic.go:356] "Generic (PLEG): container finished" podID="84bde4be-b5b2-454b-a6fd-ad3f3995d61f" containerID="eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6" exitCode=0 Jul 18 01:36:59.578928 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.578916 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" event={"ID":"84bde4be-b5b2-454b-a6fd-ad3f3995d61f","Type":"ContainerDied","Data":"eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6"} Jul 18 01:36:59.579205 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.578962 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" event={"ID":"84bde4be-b5b2-454b-a6fd-ad3f3995d61f","Type":"ContainerDied","Data":"9727f146230d1605617d2c36b87f001de32563edc07a551ee5faf73cfc4a9b79"} Jul 18 01:36:59.579205 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.578981 2585 scope.go:117] "RemoveContainer" containerID="eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6" Jul 18 01:36:59.579205 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.578933 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5" Jul 18 01:36:59.587589 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.587569 2585 scope.go:117] "RemoveContainer" containerID="eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6" Jul 18 01:36:59.587872 ip-10-0-135-148 kubenswrapper[2585]: E0718 01:36:59.587850 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6\": container with ID starting with eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6 not found: ID does not exist" containerID="eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6" Jul 18 01:36:59.587980 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.587878 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6"} err="failed to get container status \"eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6\": rpc error: code = NotFound desc = could not find container \"eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6\": container with ID starting with eb0d0b9df99091c56a2e98d8b1e2f432826c6e447ad74696ce2afc929c9d64f6 not found: ID does not exist" Jul 18 01:36:59.591514 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.591496 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be4c5b95-a792-4646-95b0-333e46581845-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq\" (UID: \"be4c5b95-a792-4646-95b0-333e46581845\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" Jul 18 01:36:59.596869 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.596845 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5"] Jul 18 01:36:59.599254 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.599233 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ad1b67/rate-test-model-ad1b67-kserve-84897c7647-t6mh5"] Jul 18 01:36:59.692543 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.692505 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be4c5b95-a792-4646-95b0-333e46581845-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq\" (UID: \"be4c5b95-a792-4646-95b0-333e46581845\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" Jul 18 01:36:59.693200 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.693180 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be4c5b95-a792-4646-95b0-333e46581845-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq\" (UID: \"be4c5b95-a792-4646-95b0-333e46581845\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" Jul 18 01:36:59.855156 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.855056 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" Jul 18 01:36:59.966619 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.966572 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" podUID="616cfb97-9357-46a7-9f85-1e3ed80f6e9a" containerName="main" containerID="cri-o://a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149" gracePeriod=45 Jul 18 01:36:59.991850 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:36:59.991804 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq"] Jul 18 01:37:00.012870 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:37:00.012822 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4c5b95_a792_4646_95b0_333e46581845.slice/crio-fe4cdceac8ffb733f653500e55b7ae3aa987a8dd7533fb34d19645122a49ae01 WatchSource:0}: Error finding container fe4cdceac8ffb733f653500e55b7ae3aa987a8dd7533fb34d19645122a49ae01: Status 404 returned error can't find the container with id fe4cdceac8ffb733f653500e55b7ae3aa987a8dd7533fb34d19645122a49ae01 Jul 18 01:37:00.071691 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.071673 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 01:37:00.220500 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.220468 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:37:00.298427 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.298399 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-home\") pod \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " Jul 18 01:37:00.298630 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.298438 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-dshm\") pod \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " Jul 18 01:37:00.298630 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.298498 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-model-cache\") pod \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " Jul 18 01:37:00.298630 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.298538 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tmp-dir\") pod \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " Jul 18 01:37:00.298630 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.298562 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtxn5\" (UniqueName: \"kubernetes.io/projected/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-kube-api-access-dtxn5\") pod \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " Jul 18 01:37:00.298630 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.298622 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tls-certs\") pod \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\" (UID: \"616cfb97-9357-46a7-9f85-1e3ed80f6e9a\") " Jul 18 01:37:00.298938 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.298769 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-home" (OuterVolumeSpecName: "home") pod "616cfb97-9357-46a7-9f85-1e3ed80f6e9a" (UID: "616cfb97-9357-46a7-9f85-1e3ed80f6e9a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:37:00.298938 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.298793 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-model-cache" (OuterVolumeSpecName: "model-cache") pod "616cfb97-9357-46a7-9f85-1e3ed80f6e9a" (UID: "616cfb97-9357-46a7-9f85-1e3ed80f6e9a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:37:00.299054 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.298946 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-model-cache\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:37:00.299054 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.298964 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-home\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:37:00.299201 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.299176 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "616cfb97-9357-46a7-9f85-1e3ed80f6e9a" (UID: "616cfb97-9357-46a7-9f85-1e3ed80f6e9a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:37:00.301089 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.301057 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-dshm" (OuterVolumeSpecName: "dshm") pod "616cfb97-9357-46a7-9f85-1e3ed80f6e9a" (UID: "616cfb97-9357-46a7-9f85-1e3ed80f6e9a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:37:00.301218 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.301122 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "616cfb97-9357-46a7-9f85-1e3ed80f6e9a" (UID: "616cfb97-9357-46a7-9f85-1e3ed80f6e9a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:37:00.301218 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.301136 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-kube-api-access-dtxn5" (OuterVolumeSpecName: "kube-api-access-dtxn5") pod "616cfb97-9357-46a7-9f85-1e3ed80f6e9a" (UID: "616cfb97-9357-46a7-9f85-1e3ed80f6e9a"). InnerVolumeSpecName "kube-api-access-dtxn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:37:00.399784 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.399758 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-dshm\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:37:00.399962 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.399787 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tmp-dir\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:37:00.399962 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.399798 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtxn5\" (UniqueName: \"kubernetes.io/projected/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-kube-api-access-dtxn5\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:37:00.399962 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.399807 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/616cfb97-9357-46a7-9f85-1e3ed80f6e9a-tls-certs\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:37:00.585045 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.585011 2585 generic.go:356] "Generic (PLEG): container finished" podID="616cfb97-9357-46a7-9f85-1e3ed80f6e9a" containerID="a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149" exitCode=0 Jul 18 01:37:00.585240 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.585059 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" Jul 18 01:37:00.585240 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.585104 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" event={"ID":"616cfb97-9357-46a7-9f85-1e3ed80f6e9a","Type":"ContainerDied","Data":"a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149"} Jul 18 01:37:00.585240 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.585151 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb" event={"ID":"616cfb97-9357-46a7-9f85-1e3ed80f6e9a","Type":"ContainerDied","Data":"16b36696c214bc6cc62ffba15d4796b3ea43902f0b8ffab53779395439a5e6bd"} Jul 18 01:37:00.585240 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.585173 2585 scope.go:117] "RemoveContainer" containerID="a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149" Jul 18 01:37:00.586471 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.586446 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" event={"ID":"be4c5b95-a792-4646-95b0-333e46581845","Type":"ContainerStarted","Data":"fe4cdceac8ffb733f653500e55b7ae3aa987a8dd7533fb34d19645122a49ae01"} Jul 18 01:37:00.594278 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.594262 2585 scope.go:117] "RemoveContainer" containerID="a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149" Jul 18 01:37:00.594519 ip-10-0-135-148 kubenswrapper[2585]: E0718 01:37:00.594502 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149\": container with ID starting with a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149 not found: ID does not exist" containerID="a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149" Jul 18 01:37:00.594589 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.594528 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149"} err="failed to get container status \"a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149\": rpc error: code = NotFound desc = could not find container \"a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149\": container with ID starting with a3ef0bccf8ffafa16fff19d0e898d6269516c6fe8cc2cd607fe966fe68d77149 not found: ID does not exist" Jul 18 01:37:00.606687 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.606658 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb"] Jul 18 01:37:00.608706 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:00.608685 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0f7fdc/rate-test-model-0f7fdc-kserve-69fd944bcf-ftwdb"] Jul 18 01:37:01.152257 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.152216 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7679-140539942320432"] Jul 18 01:37:01.152735 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.152687 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="616cfb97-9357-46a7-9f85-1e3ed80f6e9a" containerName="main" Jul 18 01:37:01.152735 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.152706 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="616cfb97-9357-46a7-9f85-1e3ed80f6e9a" containerName="main" Jul 18 01:37:01.152870 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.152822 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="616cfb97-9357-46a7-9f85-1e3ed80f6e9a" containerName="main" Jul 18 01:37:01.155707 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.155685 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-140539942320432" Jul 18 01:37:01.160541 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.160515 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zkmc5\"" Jul 18 01:37:01.166079 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.166052 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7679-140539942320432"] Jul 18 01:37:01.207331 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.207225 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-547l2\" (UniqueName: \"kubernetes.io/projected/fb43027d-fb1c-49a8-aa73-87ce3a497ee3-kube-api-access-547l2\") pod \"test-curl-7679-140539942320432\" (UID: \"fb43027d-fb1c-49a8-aa73-87ce3a497ee3\") " pod="opendatahub/test-curl-7679-140539942320432" Jul 18 01:37:01.308776 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.308739 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-547l2\" (UniqueName: \"kubernetes.io/projected/fb43027d-fb1c-49a8-aa73-87ce3a497ee3-kube-api-access-547l2\") pod \"test-curl-7679-140539942320432\" (UID: \"fb43027d-fb1c-49a8-aa73-87ce3a497ee3\") " pod="opendatahub/test-curl-7679-140539942320432" Jul 18 01:37:01.318134 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.318106 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-547l2\" (UniqueName: \"kubernetes.io/projected/fb43027d-fb1c-49a8-aa73-87ce3a497ee3-kube-api-access-547l2\") pod \"test-curl-7679-140539942320432\" (UID: \"fb43027d-fb1c-49a8-aa73-87ce3a497ee3\") " pod="opendatahub/test-curl-7679-140539942320432" Jul 18 01:37:01.457397 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.457315 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="616cfb97-9357-46a7-9f85-1e3ed80f6e9a" path="/var/lib/kubelet/pods/616cfb97-9357-46a7-9f85-1e3ed80f6e9a/volumes" Jul 18 01:37:01.457728 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.457714 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84bde4be-b5b2-454b-a6fd-ad3f3995d61f" path="/var/lib/kubelet/pods/84bde4be-b5b2-454b-a6fd-ad3f3995d61f/volumes" Jul 18 01:37:01.465406 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.465381 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-140539942320432" Jul 18 01:37:01.618175 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:01.618123 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7679-140539942320432"] Jul 18 01:37:01.621539 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:37:01.621506 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb43027d_fb1c_49a8_aa73_87ce3a497ee3.slice/crio-980c50a0bd5004ca0a6de0ec9ec7fb2f131687b17bc094b08ce9f9305cfa089f WatchSource:0}: Error finding container 980c50a0bd5004ca0a6de0ec9ec7fb2f131687b17bc094b08ce9f9305cfa089f: Status 404 returned error can't find the container with id 980c50a0bd5004ca0a6de0ec9ec7fb2f131687b17bc094b08ce9f9305cfa089f Jul 18 01:37:02.602885 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:02.602842 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7679-140539942320432" event={"ID":"fb43027d-fb1c-49a8-aa73-87ce3a497ee3","Type":"ContainerStarted","Data":"980c50a0bd5004ca0a6de0ec9ec7fb2f131687b17bc094b08ce9f9305cfa089f"} Jul 18 01:37:03.609212 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:03.609100 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" event={"ID":"be4c5b95-a792-4646-95b0-333e46581845","Type":"ContainerStarted","Data":"dfcaecb2ed0c742d9149dd18f466bfddc9324b46317b45fe4f0334df90ecf691"} Jul 18 01:37:03.610714 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:03.610685 2585 generic.go:356] "Generic (PLEG): container finished" podID="fb43027d-fb1c-49a8-aa73-87ce3a497ee3" containerID="27f6b744dac1ef614dbf1165f761932006d2c27efa9d23150708d87f42b927dd" exitCode=0 Jul 18 01:37:03.610855 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:03.610740 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7679-140539942320432" event={"ID":"fb43027d-fb1c-49a8-aa73-87ce3a497ee3","Type":"ContainerDied","Data":"27f6b744dac1ef614dbf1165f761932006d2c27efa9d23150708d87f42b927dd"} Jul 18 01:37:03.623584 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:03.623533 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" podStartSLOduration=1.6910175139999999 podStartE2EDuration="4.623518112s" totalImagesPullingTime="2.932500598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:37:03.622996715 +0000 UTC m=+3394.645131499" watchObservedRunningTime="2026-07-18 01:37:03.623518112 +0000 UTC m=+3394.645652895" Jul 18 01:37:04.616445 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:04.616410 2585 generic.go:356] "Generic (PLEG): container finished" podID="be4c5b95-a792-4646-95b0-333e46581845" containerID="dfcaecb2ed0c742d9149dd18f466bfddc9324b46317b45fe4f0334df90ecf691" exitCode=6 Jul 18 01:37:04.616985 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:04.616455 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" event={"ID":"be4c5b95-a792-4646-95b0-333e46581845","Type":"ContainerDied","Data":"dfcaecb2ed0c742d9149dd18f466bfddc9324b46317b45fe4f0334df90ecf691"} Jul 18 01:37:04.616985 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:04.616746 2585 scope.go:117] "RemoveContainer" containerID="dfcaecb2ed0c742d9149dd18f466bfddc9324b46317b45fe4f0334df90ecf691" Jul 18 01:37:04.758214 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:04.758184 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-140539942320432" Jul 18 01:37:04.779028 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:04.779001 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7679-140539942320432_fb43027d-fb1c-49a8-aa73-87ce3a497ee3/test-curl-7679-140539942320432/0.log" Jul 18 01:37:04.803389 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:04.803360 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7679-140539942320432"] Jul 18 01:37:04.807562 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:04.807536 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7679-140539942320432"] Jul 18 01:37:04.840823 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:04.840796 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-547l2\" (UniqueName: \"kubernetes.io/projected/fb43027d-fb1c-49a8-aa73-87ce3a497ee3-kube-api-access-547l2\") pod \"fb43027d-fb1c-49a8-aa73-87ce3a497ee3\" (UID: \"fb43027d-fb1c-49a8-aa73-87ce3a497ee3\") " Jul 18 01:37:04.921446 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:04.921402 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb43027d-fb1c-49a8-aa73-87ce3a497ee3-kube-api-access-547l2" (OuterVolumeSpecName: "kube-api-access-547l2") pod "fb43027d-fb1c-49a8-aa73-87ce3a497ee3" (UID: "fb43027d-fb1c-49a8-aa73-87ce3a497ee3"). InnerVolumeSpecName "kube-api-access-547l2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:37:04.941414 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:04.941388 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-547l2\" (UniqueName: \"kubernetes.io/projected/fb43027d-fb1c-49a8-aa73-87ce3a497ee3-kube-api-access-547l2\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:37:05.456611 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:05.456571 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb43027d-fb1c-49a8-aa73-87ce3a497ee3" path="/var/lib/kubelet/pods/fb43027d-fb1c-49a8-aa73-87ce3a497ee3/volumes" Jul 18 01:37:05.564458 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:05.564369 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:05.623980 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:05.623950 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-140539942320432" Jul 18 01:37:05.623980 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:05.623971 2585 scope.go:117] "RemoveContainer" containerID="27f6b744dac1ef614dbf1165f761932006d2c27efa9d23150708d87f42b927dd" Jul 18 01:37:05.626358 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:05.626291 2585 generic.go:356] "Generic (PLEG): container finished" podID="be4c5b95-a792-4646-95b0-333e46581845" containerID="f9017b1abb338e25c98889fab983d413ef160d308efedcc7e62d8f52e0e2276e" exitCode=6 Jul 18 01:37:05.626559 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:05.626329 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" event={"ID":"be4c5b95-a792-4646-95b0-333e46581845","Type":"ContainerDied","Data":"f9017b1abb338e25c98889fab983d413ef160d308efedcc7e62d8f52e0e2276e"} Jul 18 01:37:05.626699 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:05.626640 2585 scope.go:117] "RemoveContainer" containerID="f9017b1abb338e25c98889fab983d413ef160d308efedcc7e62d8f52e0e2276e" Jul 18 01:37:05.626970 ip-10-0-135-148 kubenswrapper[2585]: E0718 01:37:05.626946 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq_odh-ai-gateway-infra(be4c5b95-a792-4646-95b0-333e46581845)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" podUID="be4c5b95-a792-4646-95b0-333e46581845" Jul 18 01:37:05.633127 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:05.633107 2585 scope.go:117] "RemoveContainer" containerID="dfcaecb2ed0c742d9149dd18f466bfddc9324b46317b45fe4f0334df90ecf691" Jul 18 01:37:05.646483 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:05.646447 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:06.634970 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:06.634945 2585 scope.go:117] "RemoveContainer" containerID="f9017b1abb338e25c98889fab983d413ef160d308efedcc7e62d8f52e0e2276e" Jul 18 01:37:06.635375 ip-10-0-135-148 kubenswrapper[2585]: E0718 01:37:06.635131 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq_odh-ai-gateway-infra(be4c5b95-a792-4646-95b0-333e46581845)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" podUID="be4c5b95-a792-4646-95b0-333e46581845" Jul 18 01:37:15.420553 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:15.420516 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:19.453402 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:19.453376 2585 scope.go:117] "RemoveContainer" containerID="f9017b1abb338e25c98889fab983d413ef160d308efedcc7e62d8f52e0e2276e" Jul 18 01:37:19.694353 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:19.694319 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" event={"ID":"be4c5b95-a792-4646-95b0-333e46581845","Type":"ContainerStarted","Data":"2d055878a535f20ac8a124243ac653c2490771465fff9494c187865cdd16d275"} Jul 18 01:37:20.084236 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:20.084189 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:20.700740 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:20.700701 2585 generic.go:356] "Generic (PLEG): container finished" podID="be4c5b95-a792-4646-95b0-333e46581845" containerID="2d055878a535f20ac8a124243ac653c2490771465fff9494c187865cdd16d275" exitCode=6 Jul 18 01:37:20.701267 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:20.700780 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" event={"ID":"be4c5b95-a792-4646-95b0-333e46581845","Type":"ContainerDied","Data":"2d055878a535f20ac8a124243ac653c2490771465fff9494c187865cdd16d275"} Jul 18 01:37:20.701267 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:20.700888 2585 scope.go:117] "RemoveContainer" containerID="f9017b1abb338e25c98889fab983d413ef160d308efedcc7e62d8f52e0e2276e" Jul 18 01:37:20.701267 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:20.701169 2585 scope.go:117] "RemoveContainer" containerID="2d055878a535f20ac8a124243ac653c2490771465fff9494c187865cdd16d275" Jul 18 01:37:20.701403 ip-10-0-135-148 kubenswrapper[2585]: E0718 01:37:20.701385 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq_odh-ai-gateway-infra(be4c5b95-a792-4646-95b0-333e46581845)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" podUID="be4c5b95-a792-4646-95b0-333e46581845" Jul 18 01:37:20.719894 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:20.719866 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq"] Jul 18 01:37:21.839015 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:21.838986 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" Jul 18 01:37:21.997138 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:21.997035 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be4c5b95-a792-4646-95b0-333e46581845-maas-api-service-ca\") pod \"be4c5b95-a792-4646-95b0-333e46581845\" (UID: \"be4c5b95-a792-4646-95b0-333e46581845\") " Jul 18 01:37:21.997415 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:21.997390 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be4c5b95-a792-4646-95b0-333e46581845-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "be4c5b95-a792-4646-95b0-333e46581845" (UID: "be4c5b95-a792-4646-95b0-333e46581845"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:37:22.097790 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:22.097757 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be4c5b95-a792-4646-95b0-333e46581845-maas-api-service-ca\") on node \"ip-10-0-135-148.ec2.internal\" DevicePath \"\"" Jul 18 01:37:22.718771 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:22.718732 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" event={"ID":"be4c5b95-a792-4646-95b0-333e46581845","Type":"ContainerDied","Data":"fe4cdceac8ffb733f653500e55b7ae3aa987a8dd7533fb34d19645122a49ae01"} Jul 18 01:37:22.718771 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:22.718764 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq" Jul 18 01:37:22.719055 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:22.718821 2585 scope.go:117] "RemoveContainer" containerID="2d055878a535f20ac8a124243ac653c2490771465fff9494c187865cdd16d275" Jul 18 01:37:22.741928 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:22.741900 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq"] Jul 18 01:37:22.745705 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:22.745682 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9bd91509-2mkhq"] Jul 18 01:37:23.456535 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:23.456500 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be4c5b95-a792-4646-95b0-333e46581845" path="/var/lib/kubelet/pods/be4c5b95-a792-4646-95b0-333e46581845/volumes" Jul 18 01:37:25.986143 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:25.986112 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:26.038555 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:26.038506 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:30.738333 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:30.738289 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:30.957293 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:30.957258 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:31.592148 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:31.592113 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:31.692899 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:31.692816 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:41.511089 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:41.511056 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:46.161191 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:46.161161 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:51.981106 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:51.981064 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:52.045233 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:52.045196 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:56.802623 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:56.802577 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:57.042633 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:57.042599 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:37:57.448807 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:37:57.448760 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7784f8bd8-lvk92_d35472df-0f65-47fa-afb2-e7a8f31e8fac/authorino/0.log" Jul 18 01:38:03.003847 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:03.003794 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-29kpq_dfc0ce47-acdd-4caf-849f-4e7557ade323/manager/0.log" Jul 18 01:38:03.357389 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:03.357311 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85c6dd979f-p62qf_a8014a90-8142-4c1f-8396-917d8ab27b09/copy-manifests/0.log" Jul 18 01:38:03.385574 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:03.385535 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85c6dd979f-p62qf_a8014a90-8142-4c1f-8396-917d8ab27b09/manager/0.log" Jul 18 01:38:06.467193 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:06.467161 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7784f8bd8-lvk92_d35472df-0f65-47fa-afb2-e7a8f31e8fac/authorino/0.log" Jul 18 01:38:07.821720 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:07.821690 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f655d47ff-bmr2m_2ffaa630-baec-47ee-9418-fc56b7b206ba/kube-auth-proxy/0.log" Jul 18 01:38:08.387542 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:08.387515 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-559bddd8f8-w24db_ecfb0dac-7382-4175-804d-abd202994a71/router/0.log" Jul 18 01:38:08.824629 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:08.824598 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-kkdng_cf3016e2-a114-4579-b443-cb5429f0ce59/main/0.log" Jul 18 01:38:09.150306 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:09.150280 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-4g8l4_f19fcafc-9bf1-4f3b-b18b-2f9623874e04/main/0.log" Jul 18 01:38:09.276147 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:09.276116 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-mvfjk_fe4642bd-237b-490d-a68d-afd7aa08d952/main/0.log" Jul 18 01:38:14.505020 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:14.504984 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:38:14.603773 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:14.603740 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" limit=1024 Jul 18 01:38:17.520156 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:17.520120 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vhhcd_55cbdaea-d9a9-4eca-b1c0-65de273699dd/global-pull-secret-syncer/0.log" Jul 18 01:38:17.574596 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:17.574560 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bl9f9_4ba3c30d-3d5a-463b-918b-555fe7292755/konnectivity-agent/0.log" Jul 18 01:38:17.674703 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:17.674656 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-148.ec2.internal_ee0ba5c47453590857789475d8c354a6/haproxy/0.log" Jul 18 01:38:23.674369 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:23.674334 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7784f8bd8-lvk92_d35472df-0f65-47fa-afb2-e7a8f31e8fac/authorino/0.log" Jul 18 01:38:25.776202 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:25.776154 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-tvbdj_bdb58d5a-a4ac-435f-92ae-ff17de63bb0f/cluster-monitoring-operator/0.log" Jul 18 01:38:26.046388 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:26.046288 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rq2m9_922aee26-4b8e-4415-9e86-22eb40786d48/node-exporter/0.log" Jul 18 01:38:26.063477 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:26.063451 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rq2m9_922aee26-4b8e-4415-9e86-22eb40786d48/kube-rbac-proxy/0.log" Jul 18 01:38:26.084892 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:26.084864 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rq2m9_922aee26-4b8e-4415-9e86-22eb40786d48/init-textfile/0.log" Jul 18 01:38:27.398857 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.398809 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8"] Jul 18 01:38:27.399277 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.399180 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be4c5b95-a792-4646-95b0-333e46581845" containerName="revoke-keys" Jul 18 01:38:27.399277 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.399192 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="be4c5b95-a792-4646-95b0-333e46581845" containerName="revoke-keys" Jul 18 01:38:27.399277 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.399209 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be4c5b95-a792-4646-95b0-333e46581845" containerName="revoke-keys" Jul 18 01:38:27.399277 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.399214 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="be4c5b95-a792-4646-95b0-333e46581845" containerName="revoke-keys" Jul 18 01:38:27.399277 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.399223 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be4c5b95-a792-4646-95b0-333e46581845" containerName="revoke-keys" Jul 18 01:38:27.399277 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.399229 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="be4c5b95-a792-4646-95b0-333e46581845" containerName="revoke-keys" Jul 18 01:38:27.399277 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.399238 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb43027d-fb1c-49a8-aa73-87ce3a497ee3" containerName="test-curl-7679-140539942320432" Jul 18 01:38:27.399277 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.399244 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb43027d-fb1c-49a8-aa73-87ce3a497ee3" containerName="test-curl-7679-140539942320432" Jul 18 01:38:27.399733 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.399305 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb43027d-fb1c-49a8-aa73-87ce3a497ee3" containerName="test-curl-7679-140539942320432" Jul 18 01:38:27.399733 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.399316 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="be4c5b95-a792-4646-95b0-333e46581845" containerName="revoke-keys" Jul 18 01:38:27.399733 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.399324 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="be4c5b95-a792-4646-95b0-333e46581845" containerName="revoke-keys" Jul 18 01:38:27.402546 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.402528 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.405038 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.405015 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gb6px\"/\"default-dockercfg-hr289\"" Jul 18 01:38:27.405165 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.405035 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gb6px\"/\"openshift-service-ca.crt\"" Jul 18 01:38:27.405970 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.405944 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gb6px\"/\"kube-root-ca.crt\"" Jul 18 01:38:27.408950 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.408930 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8"] Jul 18 01:38:27.467011 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.466974 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-proc\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.467011 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.467014 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbkm4\" (UniqueName: \"kubernetes.io/projected/71f677a5-4ec2-4159-a585-b375d320482b-kube-api-access-bbkm4\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.467265 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.467066 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-podres\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.467265 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.467108 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-lib-modules\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.467386 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.467274 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-sys\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.568724 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.568683 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-sys\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.568951 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.568751 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-proc\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.568951 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.568775 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbkm4\" (UniqueName: \"kubernetes.io/projected/71f677a5-4ec2-4159-a585-b375d320482b-kube-api-access-bbkm4\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.568951 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.568805 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-podres\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.568951 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.568825 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-proc\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.568951 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.568853 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-lib-modules\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.569166 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.568953 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-podres\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.569166 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.568852 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-sys\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.569166 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.568970 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/71f677a5-4ec2-4159-a585-b375d320482b-lib-modules\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.577534 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.577502 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbkm4\" (UniqueName: \"kubernetes.io/projected/71f677a5-4ec2-4159-a585-b375d320482b-kube-api-access-bbkm4\") pod \"perf-node-gather-daemonset-ckkh8\" (UID: \"71f677a5-4ec2-4159-a585-b375d320482b\") " pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.683301 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.683208 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-kdk54_70547afb-1be6-4358-b0cd-5349d7dbf273/networking-console-plugin/0.log" Jul 18 01:38:27.713661 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.713621 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:27.846918 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:27.846852 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8"] Jul 18 01:38:27.849600 ip-10-0-135-148 kubenswrapper[2585]: W0718 01:38:27.849569 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod71f677a5_4ec2_4159_a585_b375d320482b.slice/crio-7c6248f6a0035092aeef91b236951095966ec547774c1b1529b53c6b94a499e8 WatchSource:0}: Error finding container 7c6248f6a0035092aeef91b236951095966ec547774c1b1529b53c6b94a499e8: Status 404 returned error can't find the container with id 7c6248f6a0035092aeef91b236951095966ec547774c1b1529b53c6b94a499e8 Jul 18 01:38:28.006650 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:28.006561 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" event={"ID":"71f677a5-4ec2-4159-a585-b375d320482b","Type":"ContainerStarted","Data":"f541a51c00b1d82329cd7385f8ba96e8a90bb2764ecd869f30f871092a416a91"} Jul 18 01:38:28.006650 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:28.006612 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" event={"ID":"71f677a5-4ec2-4159-a585-b375d320482b","Type":"ContainerStarted","Data":"7c6248f6a0035092aeef91b236951095966ec547774c1b1529b53c6b94a499e8"} Jul 18 01:38:28.006954 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:28.006703 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:28.021786 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:28.021732 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" podStartSLOduration=1.021716764 podStartE2EDuration="1.021716764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:38:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:38:28.020481701 +0000 UTC m=+3479.042616487" watchObservedRunningTime="2026-07-18 01:38:28.021716764 +0000 UTC m=+3479.043851547" Jul 18 01:38:28.268498 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:28.268408 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/1.log" Jul 18 01:38:28.276419 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:28.276388 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kgsmq_15ccd479-3004-4964-a298-0c510fc4bf9b/console-operator/2.log" Jul 18 01:38:29.407139 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:29.407105 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-9w9q5_e074f31c-0e2d-4e42-b988-2564e4e2d6a0/volume-data-source-validator/0.log" Jul 18 01:38:30.408220 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:30.408180 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2gm4c_f12d587d-fdc5-4173-a8a3-a066a8f8b848/dns/0.log" Jul 18 01:38:30.427850 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:30.427802 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2gm4c_f12d587d-fdc5-4173-a8a3-a066a8f8b848/kube-rbac-proxy/0.log" Jul 18 01:38:30.575170 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:30.575137 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mfx87_4a9721fd-fe3c-4f2d-8d09-24a59fc24ac3/dns-node-resolver/0.log" Jul 18 01:38:31.150918 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:31.150882 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-67ccfcbdb9-w4t85_022ce19a-fe97-4101-bda1-2bf053b10ca2/registry/0.log" Jul 18 01:38:31.180944 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:31.180911 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cxmc9_d800e91f-1436-4859-83e2-9bd813856736/node-ca/0.log" Jul 18 01:38:32.309879 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:32.309850 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f655d47ff-bmr2m_2ffaa630-baec-47ee-9418-fc56b7b206ba/kube-auth-proxy/0.log" Jul 18 01:38:32.481028 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:32.480992 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-559bddd8f8-w24db_ecfb0dac-7382-4175-804d-abd202994a71/router/0.log" Jul 18 01:38:33.149215 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:33.149168 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8n9md_80a66f7f-3a1d-4252-8139-846b74939b67/serve-healthcheck-canary/0.log" Jul 18 01:38:33.665772 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:33.665739 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-z987w_602eb3d3-135b-46da-8f9a-0061de2cc17b/insights-operator/0.log" Jul 18 01:38:33.667528 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:33.667504 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-z987w_602eb3d3-135b-46da-8f9a-0061de2cc17b/insights-operator/1.log" Jul 18 01:38:33.813129 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:33.813079 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f2qcd_cc428557-f751-4d98-abc8-7d1e12a09593/kube-rbac-proxy/0.log" Jul 18 01:38:33.832233 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:33.832208 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f2qcd_cc428557-f751-4d98-abc8-7d1e12a09593/exporter/0.log" Jul 18 01:38:33.851708 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:33.851685 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f2qcd_cc428557-f751-4d98-abc8-7d1e12a09593/extractor/0.log" Jul 18 01:38:34.021251 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:34.021170 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gb6px/perf-node-gather-daemonset-ckkh8" Jul 18 01:38:36.704511 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:36.704432 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-29kpq_dfc0ce47-acdd-4caf-849f-4e7557ade323/manager/0.log" Jul 18 01:38:36.918645 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:36.918613 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85c6dd979f-p62qf_a8014a90-8142-4c1f-8396-917d8ab27b09/manager/0.log" Jul 18 01:38:36.939052 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:36.939027 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85c6dd979f-p62qf_a8014a90-8142-4c1f-8396-917d8ab27b09/copy-manifests/0.log" Jul 18 01:38:38.291630 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:38.291599 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b88d5d556-mksdh_c103c660-6c37-4855-8aad-ad0557eb272a/manager/0.log" Jul 18 01:38:38.424891 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:38.424861 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f655d47ff-bmr2m_2ffaa630-baec-47ee-9418-fc56b7b206ba/kube-auth-proxy/0.log" Jul 18 01:38:38.575944 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:38.575855 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-559bddd8f8-w24db_ecfb0dac-7382-4175-804d-abd202994a71/router/0.log" Jul 18 01:38:39.889215 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:39.889180 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f655d47ff-bmr2m_2ffaa630-baec-47ee-9418-fc56b7b206ba/kube-auth-proxy/0.log" Jul 18 01:38:40.032666 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:40.032636 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-559bddd8f8-w24db_ecfb0dac-7382-4175-804d-abd202994a71/router/0.log" Jul 18 01:38:43.725358 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:43.725317 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-hdjjz_2f6b75f9-43a8-4d7c-9cd0-7a350a753397/kube-storage-version-migrator-operator/1.log" Jul 18 01:38:43.727515 ip-10-0-135-148 kubenswrapper[2585]: I0718 01:38:43.727480 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-hdjjz_2f6b75f9-43a8-4d7c-9cd0-7a350a753397/kube-storage-version-migrator-operator/0.log"