Jul 06 23:40:46.843924 ip-10-0-135-66 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 23:40:46.843935 ip-10-0-135-66 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 23:40:46.843942 ip-10-0-135-66 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 23:40:46.844199 ip-10-0-135-66 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 23:40:57.014701 ip-10-0-135-66 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 23:40:57.014722 ip-10-0-135-66 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot cd6e09b5eabc4e0480e326ed07338b1d -- Jul 06 23:42:50.994960 ip-10-0-135-66 systemd[1]: Starting Kubernetes Kubelet... Jul 06 23:42:51.357960 ip-10-0-135-66 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:42:51.357960 ip-10-0-135-66 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 23:42:51.357960 ip-10-0-135-66 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:42:51.357960 ip-10-0-135-66 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 23:42:51.357960 ip-10-0-135-66 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:42:51.360498 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.360329 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 23:42:51.367626 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367587 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:42:51.367626 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367620 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:42:51.367626 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367624 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:42:51.367626 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367627 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:42:51.367626 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367630 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:42:51.367626 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367633 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:42:51.367626 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367636 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:42:51.367626 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367639 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:42:51.367626 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367642 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:42:51.367626 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367645 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367649 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367654 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367657 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367660 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367662 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367665 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367669 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367671 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367674 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367677 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367679 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367682 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367685 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367689 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367691 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367694 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367696 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367699 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367701 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:42:51.368002 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367704 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367706 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367711 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367716 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367719 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367723 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367726 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367728 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367731 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367734 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367737 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367740 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367742 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367745 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367748 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367751 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367754 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367757 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367759 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367762 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:42:51.368522 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367764 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367767 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367769 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367772 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367774 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367776 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367780 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367784 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367786 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367789 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367792 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367796 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367800 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367802 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367805 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367808 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367810 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367813 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367815 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:42:51.369045 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367818 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367820 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367823 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367825 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367829 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367831 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367833 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367836 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367838 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367841 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367843 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367845 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367848 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367850 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367853 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367860 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367863 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367865 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367867 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367870 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:42:51.369499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367872 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367875 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367877 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367879 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367882 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367884 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367887 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367889 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367892 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367894 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.367897 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368685 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368694 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368697 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368700 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368703 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368705 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368708 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368711 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368714 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:42:51.370030 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368716 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368719 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368722 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368724 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368727 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368729 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368732 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368734 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368738 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368741 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368743 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368746 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368748 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368751 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368753 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368756 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368758 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368761 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368764 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368766 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368769 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:42:51.370517 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368771 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368775 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368779 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368783 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368786 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368789 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368793 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368795 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368798 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368801 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368803 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368805 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368808 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368810 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368813 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368815 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368818 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368821 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368824 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:42:51.371098 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368827 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368831 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368834 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368836 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368846 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368848 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368852 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368856 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368859 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368862 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368865 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368868 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368870 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368873 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368876 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368879 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368881 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368884 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368887 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:42:51.371592 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368890 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368892 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368895 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368897 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368900 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368903 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368906 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368908 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368911 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368913 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368916 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368918 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368921 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368923 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368925 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368929 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368933 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368936 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368939 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368942 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:42:51.372062 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368944 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368947 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368949 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368952 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368954 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368956 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368959 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368961 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368964 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368967 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.368969 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369373 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369378 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369381 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369384 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369387 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369389 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369392 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369394 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:42:51.372608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369397 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369399 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369402 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369405 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369408 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369410 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369413 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369415 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369418 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369421 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369424 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369426 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369429 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369431 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369433 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369438 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369441 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369445 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369448 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:42:51.373080 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369451 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369454 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369457 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369460 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369463 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369466 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369469 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369472 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369475 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369478 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369481 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369484 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369486 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369488 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369490 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369493 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369495 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369498 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369500 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369503 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:42:51.373535 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369506 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369509 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369512 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369515 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369518 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369521 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369523 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369526 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369528 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369531 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369534 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369536 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369539 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369542 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369544 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369547 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369549 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369552 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369555 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369558 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:42:51.374084 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369560 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369563 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369581 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369586 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369589 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369591 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369594 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369596 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369599 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369601 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369604 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369607 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369610 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369612 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369614 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369618 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369621 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369623 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369626 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369628 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:42:51.374584 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369631 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369633 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369635 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369638 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369641 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369643 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369646 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369648 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369651 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369656 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369658 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.369661 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369749 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369758 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369765 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369770 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369774 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369778 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369782 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369786 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369789 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 23:42:51.375125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369792 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369796 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369799 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369802 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369805 2560 flags.go:64] FLAG: --cgroup-root="" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369808 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369811 2560 flags.go:64] FLAG: --client-ca-file="" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369815 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369818 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369822 2560 flags.go:64] FLAG: --cluster-domain="" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369825 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369828 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369830 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369834 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369838 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369842 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369845 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369848 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369851 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369855 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369860 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369863 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369866 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369870 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 23:42:51.375661 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369873 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369876 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369879 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369882 2560 flags.go:64] FLAG: --enable-server="true" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369885 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369890 2560 flags.go:64] FLAG: --event-burst="100" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369893 2560 flags.go:64] FLAG: --event-qps="50" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369895 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369899 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369901 2560 flags.go:64] FLAG: --eviction-hard="" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369905 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369908 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369911 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369914 2560 flags.go:64] FLAG: --eviction-soft="" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369917 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369919 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369924 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369927 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369930 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369933 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369935 2560 flags.go:64] FLAG: --feature-gates="" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369941 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369944 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369947 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369950 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369953 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 06 23:42:51.376225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369956 2560 flags.go:64] FLAG: --help="false" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369959 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369963 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369966 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369969 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369973 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369976 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369979 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369982 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369985 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369987 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369990 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369993 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369996 2560 flags.go:64] FLAG: --kube-reserved="" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.369999 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370001 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370004 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370007 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370010 2560 flags.go:64] FLAG: --lock-file="" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370013 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370017 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370020 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370026 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 23:42:51.377031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370030 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370033 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370035 2560 flags.go:64] FLAG: --logging-format="text" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370039 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370042 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370045 2560 flags.go:64] FLAG: --manifest-url="" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370048 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370052 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370055 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370059 2560 flags.go:64] FLAG: --max-pods="110" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370062 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370066 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370069 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370072 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370075 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370078 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370080 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370088 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370091 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370094 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370097 2560 flags.go:64] FLAG: --pod-cidr="" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370100 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370106 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370109 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 23:42:51.377704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370112 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370114 2560 flags.go:64] FLAG: --port="10250" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370117 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370120 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-078093458ed21abde" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370123 2560 flags.go:64] FLAG: --qos-reserved="" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370127 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370130 2560 flags.go:64] FLAG: --register-node="true" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370132 2560 flags.go:64] FLAG: --register-with-taints="" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370136 2560 flags.go:64] FLAG: --registry-burst="10" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370140 2560 flags.go:64] FLAG: --registry-qps="5" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370143 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370145 2560 flags.go:64] FLAG: --reserved-memory="" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370149 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370152 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370155 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370158 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370160 2560 flags.go:64] FLAG: --runonce="false" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370163 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370166 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370169 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370172 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370175 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370178 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370181 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370184 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370186 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 23:42:51.378332 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370189 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370192 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370195 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370198 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370201 2560 flags.go:64] FLAG: --system-cgroups="" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370203 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370209 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370212 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370214 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370219 2560 flags.go:64] FLAG: --tls-min-version="" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370221 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370225 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370228 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370231 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370234 2560 flags.go:64] FLAG: --v="2" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370238 2560 flags.go:64] FLAG: --version="false" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370246 2560 flags.go:64] FLAG: --vmodule="" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370250 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.370253 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370378 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370381 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370384 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370387 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:42:51.379044 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370389 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370392 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370395 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370397 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370400 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370402 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370405 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370407 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370410 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370412 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370415 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370417 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370420 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370422 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370425 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370428 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370430 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370433 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370435 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370437 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:42:51.379657 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370442 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370444 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370447 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370450 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370453 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370456 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370459 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370462 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370465 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370467 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370471 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370473 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370476 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370478 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370481 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370483 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370490 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370492 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370495 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:42:51.380152 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370497 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370499 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370502 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370504 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370507 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370510 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370512 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370514 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370516 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370519 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370522 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370524 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370527 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370531 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370534 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370536 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370539 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370541 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370544 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:42:51.380706 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370546 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370551 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370557 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370561 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370564 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370584 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370589 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370593 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370597 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370600 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370602 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370605 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370608 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370611 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370614 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370616 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370619 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370621 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370624 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:42:51.381173 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370627 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370629 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370632 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370634 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370637 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370639 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370642 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370645 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370648 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370650 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370653 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370655 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370658 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370660 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370663 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370668 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370670 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:42:51.381663 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.370673 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.371372 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.379038 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.379063 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.379086 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.379097 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379158 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379164 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379167 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379171 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379175 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379178 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379180 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379183 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379186 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:42:51.382082 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379189 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379191 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379193 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379196 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379199 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379203 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379207 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379210 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379213 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379216 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379219 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379221 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379224 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379226 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379229 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379232 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379234 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379236 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379241 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379245 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:42:51.382471 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379249 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379252 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379254 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379257 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379259 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379262 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379264 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379267 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379270 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379272 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379275 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379277 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379279 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379282 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379284 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379287 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379289 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379292 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379294 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379297 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:42:51.382991 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379299 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379302 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379304 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379306 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379309 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379311 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379314 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379316 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379318 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379321 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379324 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379327 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379331 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379334 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379337 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379339 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379342 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379345 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379347 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:42:51.383510 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379349 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379352 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379354 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379357 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379359 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379361 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379364 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379366 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379369 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379371 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379373 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379376 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379378 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379380 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379383 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379385 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379387 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379390 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379392 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:42:51.384000 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379394 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379397 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379400 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379402 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379405 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379408 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379411 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379414 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379417 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379421 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379424 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379427 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.379433 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379561 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379582 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:42:51.384553 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379587 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379589 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379593 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379595 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379599 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379601 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379605 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379608 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379611 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379613 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379617 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379620 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379623 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379625 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379628 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379630 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379634 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379638 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379641 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:42:51.384985 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379644 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379646 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379649 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379652 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379655 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379659 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379662 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379666 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379669 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379673 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379676 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379679 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379682 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379685 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379688 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379690 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379693 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379695 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379698 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379700 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:42:51.385538 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379703 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379705 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379708 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379710 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379713 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379716 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379718 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379721 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379723 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379726 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379728 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379730 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379733 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379735 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379738 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379741 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379744 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379747 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379751 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379754 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:42:51.386075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379758 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379761 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379763 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379766 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379769 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379772 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379774 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379776 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379779 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379781 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379784 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379786 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379788 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379791 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379794 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379796 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379799 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379801 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379804 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:42:51.386830 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379806 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379809 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379811 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379814 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379816 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379818 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379821 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379824 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379826 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379829 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379832 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379835 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379837 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379840 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379843 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379846 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379848 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379851 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:42:51.387608 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:51.379853 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:42:51.388095 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.379858 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:42:51.388095 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.380010 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 23:42:51.388095 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.382840 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 23:42:51.388095 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.384308 2560 server.go:1019] "Starting client certificate rotation" Jul 06 23:42:51.388095 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.384428 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:42:51.388095 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.384484 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:42:51.388095 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.386793 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:42:51.388638 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.388552 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:42:51.403820 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.403768 2560 log.go:25] "Validated CRI v1 runtime API" Jul 06 23:42:51.406902 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.406851 2560 log.go:25] "Validated CRI v1 image API" Jul 06 23:42:51.408651 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.408605 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 23:42:51.411856 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.411819 2560 fs.go:135] Filesystem UUIDs: map[79dda64f-04bf-4035-baa3-4d748d95d722:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a95d53f9-fcf0-46f0-b0a6-9cdf677a092a:/dev/nvme0n1p4] Jul 06 23:42:51.411957 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.411853 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 23:42:51.415525 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.415502 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:42:51.417715 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.417595 2560 manager.go:216] Machine: {Timestamp:2026-07-06 23:42:51.415650483 +0000 UTC m=+0.307839239 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100549 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f6d8840c8a06b01c2b8c99151f335 SystemUUID:ec2f6d88-40c8-a06b-01c2-b8c99151f335 BootID:cd6e09b5-eabc-4e04-80e3-26ed07338b1d Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:23:07:b5:f2:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:23:07:b5:f2:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:58:63:02:de:01 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 23:42:51.417715 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.417712 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 23:42:51.417834 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.417810 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 23:42:51.421081 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.421038 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 23:42:51.421250 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.421083 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-66.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 23:42:51.421835 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.421823 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 23:42:51.421867 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.421837 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 23:42:51.421867 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.421847 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:42:51.421867 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.421862 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:42:51.423406 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.423390 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 23:42:51.427675 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.427656 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:42:51.427830 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.427819 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 23:42:51.427866 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.427859 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 06 23:42:51.427893 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.427875 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 23:42:51.427893 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.427891 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 23:42:51.427958 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.427901 2560 kubelet.go:398] "Adding apiserver pod source" Jul 06 23:42:51.427958 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.427912 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 23:42:51.428432 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.428419 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:42:51.428487 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.428439 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:42:51.430670 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.430646 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 23:42:51.432719 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.432704 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 23:42:51.432774 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.432737 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 23:42:51.434001 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.433987 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 23:42:51.434062 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434012 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 23:42:51.434062 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434022 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 23:42:51.434062 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434030 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 23:42:51.434062 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434044 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 23:42:51.434062 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434053 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 23:42:51.434207 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434066 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 23:42:51.434207 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434071 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 23:42:51.434207 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434081 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 23:42:51.434207 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434088 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 23:42:51.434207 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434097 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 23:42:51.434207 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434103 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 23:42:51.434786 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434773 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 23:42:51.434838 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.434789 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 23:42:51.438372 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.438352 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kxm5r" Jul 06 23:42:51.438600 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.438587 2560 server.go:1268] "Started kubelet" Jul 06 23:42:51.438714 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.438681 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 23:42:51.439487 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.439416 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 23:42:51.439624 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.439519 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 23:42:51.439783 ip-10-0-135-66 systemd[1]: Started Kubernetes Kubelet. Jul 06 23:42:51.440092 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.440067 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-66.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 23:42:51.440234 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.440212 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 23:42:51.440316 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.440241 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-66.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 23:42:51.440898 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.440796 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 06 23:42:51.441075 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.441059 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 23:42:51.444900 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.444868 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 23:42:51.445835 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.445819 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kxm5r" Jul 06 23:42:51.446603 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.445701 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-66.ec2.internal.18bfd7e897a48bcb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-66.ec2.internal,UID:ip-10-0-135-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-66.ec2.internal,},FirstTimestamp:2026-07-06 23:42:51.438533579 +0000 UTC m=+0.330722248,LastTimestamp:2026-07-06 23:42:51.438533579 +0000 UTC m=+0.330722248,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-66.ec2.internal,}" Jul 06 23:42:51.450543 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.450513 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 23:42:51.450684 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.450518 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 23:42:51.450833 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.450797 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:51.451298 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.451279 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 23:42:51.451425 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.451309 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 23:42:51.451502 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.451493 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 23:42:51.451678 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.451659 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 06 23:42:51.451678 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.451673 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 06 23:42:51.452088 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.452071 2560 factory.go:153] Registering CRI-O factory Jul 06 23:42:51.452088 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.452088 2560 factory.go:223] Registration of the crio container factory successfully Jul 06 23:42:51.452216 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.452143 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 23:42:51.452216 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.452153 2560 factory.go:55] Registering systemd factory Jul 06 23:42:51.452216 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.452161 2560 factory.go:223] Registration of the systemd container factory successfully Jul 06 23:42:51.452216 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.452185 2560 factory.go:103] Registering Raw factory Jul 06 23:42:51.452216 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.452201 2560 manager.go:1196] Started watching for new ooms in manager Jul 06 23:42:51.452908 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.452891 2560 manager.go:349] Starting recovery of all containers Jul 06 23:42:51.455102 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.455083 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:42:51.458480 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.458450 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-66.ec2.internal\" not found" node="ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.464471 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.464290 2560 manager.go:354] Recovery completed Jul 06 23:42:51.469145 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.469126 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:42:51.472250 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.472231 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:42:51.472335 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.472264 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:42:51.472335 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.472275 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:42:51.473020 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.473003 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:42:51.473398 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.473385 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 23:42:51.473398 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.473397 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 23:42:51.473487 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.473415 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:42:51.473851 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.473833 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:42:51.473936 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.473861 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:42:51.473936 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.473875 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:42:51.475613 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.475598 2560 policy_none.go:49] "None policy: Start" Jul 06 23:42:51.475685 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.475626 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 23:42:51.475685 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.475641 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 23:42:51.477673 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.477658 2560 policy_none.go:47] "Start" Jul 06 23:42:51.520817 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.520798 2560 manager.go:337] "Starting Device Plugin manager" Jul 06 23:42:51.520951 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.520851 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 23:42:51.520951 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.520866 2560 server.go:87] "Starting device plugin registration server" Jul 06 23:42:51.521213 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.521200 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 23:42:51.521261 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.521216 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 23:42:51.521319 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.521299 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 23:42:51.521407 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.521393 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 23:42:51.521407 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.521407 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 23:42:51.522049 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.522022 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 23:42:51.522152 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.522074 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:51.554989 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.554943 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 23:42:51.560126 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.556470 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 23:42:51.560126 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.556511 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 23:42:51.560126 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.556535 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 23:42:51.560126 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.556594 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 23:42:51.560126 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.558928 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:42:51.622211 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.622097 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:42:51.623180 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.623161 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:42:51.623180 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.623193 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:42:51.623339 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.623204 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:42:51.623339 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.623229 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.631043 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.631019 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.631128 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.631051 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-66.ec2.internal\": node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:51.646321 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.646288 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:51.656930 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.656898 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-66.ec2.internal"] Jul 06 23:42:51.657064 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.656978 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:42:51.659247 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.659230 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:42:51.659316 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.659265 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:42:51.659316 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.659277 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:42:51.661668 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.661654 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:42:51.661784 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.661766 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.661822 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.661803 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:42:51.662564 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.662548 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:42:51.662674 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.662552 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:42:51.662674 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.662605 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:42:51.662674 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.662618 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:42:51.662674 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.662615 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:42:51.662674 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.662635 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:42:51.665082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.665064 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.665162 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.665095 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:42:51.665973 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.665957 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:42:51.666052 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.665986 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:42:51.666052 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.665995 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:42:51.692776 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.692744 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-66.ec2.internal\" not found" node="ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.696883 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.696859 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-66.ec2.internal\" not found" node="ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.746432 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.746393 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:51.753363 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.753337 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/891180d0e757f7b45d73ab1d4d4824ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal\" (UID: \"891180d0e757f7b45d73ab1d4d4824ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.753428 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.753368 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/891180d0e757f7b45d73ab1d4d4824ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal\" (UID: \"891180d0e757f7b45d73ab1d4d4824ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.753428 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.753385 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33725b36b540b2d810e88a2177f8bbbe-config\") pod \"kube-apiserver-proxy-ip-10-0-135-66.ec2.internal\" (UID: \"33725b36b540b2d810e88a2177f8bbbe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.847554 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.847506 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:51.853926 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.853890 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/891180d0e757f7b45d73ab1d4d4824ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal\" (UID: \"891180d0e757f7b45d73ab1d4d4824ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.854035 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.853936 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/891180d0e757f7b45d73ab1d4d4824ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal\" (UID: \"891180d0e757f7b45d73ab1d4d4824ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.854035 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.853960 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33725b36b540b2d810e88a2177f8bbbe-config\") pod \"kube-apiserver-proxy-ip-10-0-135-66.ec2.internal\" (UID: \"33725b36b540b2d810e88a2177f8bbbe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.854035 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.854007 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/891180d0e757f7b45d73ab1d4d4824ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal\" (UID: \"891180d0e757f7b45d73ab1d4d4824ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.854141 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.854010 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33725b36b540b2d810e88a2177f8bbbe-config\") pod \"kube-apiserver-proxy-ip-10-0-135-66.ec2.internal\" (UID: \"33725b36b540b2d810e88a2177f8bbbe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.854141 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.854014 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/891180d0e757f7b45d73ab1d4d4824ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal\" (UID: \"891180d0e757f7b45d73ab1d4d4824ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.948463 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:51.948360 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:51.995889 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.995852 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" Jul 06 23:42:51.999914 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:51.999885 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-66.ec2.internal" Jul 06 23:42:52.048681 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:52.048642 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:52.149196 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:52.149161 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:52.249871 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:52.249820 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:52.350266 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:52.350208 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:52.383259 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.383219 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 23:42:52.383897 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.383439 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 23:42:52.383897 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.383437 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 23:42:52.451274 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:52.451222 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:52.452060 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.452032 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 23:37:51 +0000 UTC" deadline="2028-03-11 14:52:10.73458836 +0000 UTC" Jul 06 23:42:52.452135 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.452063 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14727h9m18.282530897s" Jul 06 23:42:52.552522 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:52.552423 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:52.606122 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:52.606084 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod891180d0e757f7b45d73ab1d4d4824ae.slice/crio-34122de97745f4db1fc6b5222624dd118405e6e9b6c2b6a007f0dd6dbdcee8c9 WatchSource:0}: Error finding container 34122de97745f4db1fc6b5222624dd118405e6e9b6c2b6a007f0dd6dbdcee8c9: Status 404 returned error can't find the container with id 34122de97745f4db1fc6b5222624dd118405e6e9b6c2b6a007f0dd6dbdcee8c9 Jul 06 23:42:52.609899 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:52.609863 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33725b36b540b2d810e88a2177f8bbbe.slice/crio-7ec3ca7dfb1bd515cccc57bf4f241ce55c0b6d43d1c5f81f2a501a066b38d070 WatchSource:0}: Error finding container 7ec3ca7dfb1bd515cccc57bf4f241ce55c0b6d43d1c5f81f2a501a066b38d070: Status 404 returned error can't find the container with id 7ec3ca7dfb1bd515cccc57bf4f241ce55c0b6d43d1c5f81f2a501a066b38d070 Jul 06 23:42:52.612883 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.612863 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:42:52.652929 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:52.652879 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-66.ec2.internal\" not found" Jul 06 23:42:52.705753 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.705708 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:42:52.751595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.751515 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" Jul 06 23:42:52.764350 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.764307 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:42:52.766189 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.766165 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-66.ec2.internal" Jul 06 23:42:52.775391 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.775357 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:42:52.791522 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:52.791492 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:42:53.308591 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.308539 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:42:53.405445 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.405402 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:42:53.429812 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.429775 2560 apiserver.go:52] "Watching apiserver" Jul 06 23:42:53.437622 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.437582 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 23:42:53.439983 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.439062 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-66.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg","openshift-multus/multus-additional-cni-plugins-tk9hj","openshift-multus/network-metrics-daemon-f2vgw","openshift-network-diagnostics/network-check-target-vlbd8","openshift-ovn-kubernetes/ovnkube-node-92dbw","kube-system/global-pull-secret-syncer-nng68","openshift-cluster-node-tuning-operator/tuned-zjxbk","openshift-dns/node-resolver-qcg78","openshift-image-registry/node-ca-kbskp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal","openshift-multus/multus-552zh","openshift-network-operator/iptables-alerter-hmzwd","kube-system/konnectivity-agent-6l7np"] Jul 06 23:42:53.442335 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.442299 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:53.442491 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:53.442423 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:42:53.444649 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.444620 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:42:53.447679 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.447208 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 23:42:53.447679 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.447448 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sn7x6\"" Jul 06 23:42:53.447679 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.447480 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 23:42:53.447679 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.447675 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.449712 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.449683 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:53.449862 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:53.449765 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:42:53.449948 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.449919 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 23:42:53.450265 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.450244 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 23:42:53.450356 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.450282 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 23:42:53.450418 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.450351 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 23:42:53.450418 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.450366 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8gz6t\"" Jul 06 23:42:53.450418 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.450280 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 23:42:53.451829 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.451806 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 23:42:53.452842 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.452821 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:42:53.452958 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:53.452893 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:42:53.455757 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.455639 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.459028 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.458999 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 23:42:53.459567 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.459540 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.459835 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.459805 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 23:42:53.459944 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.459878 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 23:42:53.459944 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.459869 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 23:42:53.460413 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.460378 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 23:42:53.460704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.460676 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vfjdm\"" Jul 06 23:42:53.460822 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.460735 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 23:42:53.462170 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.461772 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qcg78" Jul 06 23:42:53.463005 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.462983 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:42:53.463107 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463025 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-kubelet\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463107 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463062 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a34c708d-ebcf-49e2-ad65-a18ece0d1858-agent-certs\") pod \"konnectivity-agent-6l7np\" (UID: \"a34c708d-ebcf-49e2-ad65-a18ece0d1858\") " pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:42:53.463107 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463095 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-os-release\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.463302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463108 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 23:42:53.463302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463123 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.463302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463149 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a524114-7358-43de-85d9-00e55e281c8a-dbus\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:53.463302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463179 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-etc-openvswitch\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463205 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-log-socket\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463229 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-system-cni-dir\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.463302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463268 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s62bx\"" Jul 06 23:42:53.463302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463271 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2db27634-076c-484d-a1bf-9633194d39c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.463302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463304 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kcjv\" (UniqueName: \"kubernetes.io/projected/2db27634-076c-484d-a1bf-9633194d39c6-kube-api-access-9kcjv\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463333 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/753ab644-3d33-4875-bf8d-509d4451c8da-env-overrides\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463349 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/753ab644-3d33-4875-bf8d-509d4451c8da-ovn-node-metrics-cert\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463408 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-systemd-units\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463437 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-cni-bin\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463455 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-cni-netd\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463498 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463536 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/753ab644-3d33-4875-bf8d-509d4451c8da-ovnkube-config\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463565 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29gjk\" (UniqueName: \"kubernetes.io/projected/58cbc7f0-0b10-46b4-8f20-73034d089bc9-kube-api-access-29gjk\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463674 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-slash\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463704 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-run-openvswitch\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463737 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-node-log\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.463772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463772 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-run-ovn-kubernetes\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463799 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/753ab644-3d33-4875-bf8d-509d4451c8da-ovnkube-script-lib\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463844 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-cnibin\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463888 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmdtd\" (UniqueName: \"kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd\") pod \"network-check-target-vlbd8\" (UID: \"d5222093-6d7d-46d1-93f1-47245a02ae3c\") " pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463915 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-run-systemd\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463954 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-run-ovn\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.463988 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464011 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2db27634-076c-484d-a1bf-9633194d39c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464039 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a524114-7358-43de-85d9-00e55e281c8a-kubelet-config\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464087 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464119 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-run-netns\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464145 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-var-lib-openvswitch\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464190 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a34c708d-ebcf-49e2-ad65-a18ece0d1858-konnectivity-ca\") pod \"konnectivity-agent-6l7np\" (UID: \"a34c708d-ebcf-49e2-ad65-a18ece0d1858\") " pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb2p9\" (UniqueName: \"kubernetes.io/projected/753ab644-3d33-4875-bf8d-509d4451c8da-kube-api-access-gb2p9\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464255 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:53.464343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2db27634-076c-484d-a1bf-9633194d39c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.465129 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464362 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 23:42:53.465129 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464598 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kbskp" Jul 06 23:42:53.465129 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464661 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bljkr\"" Jul 06 23:42:53.465129 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.464954 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:42:53.466945 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.466926 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 23:42:53.467948 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.467926 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 23:42:53.468194 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.468174 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 23:42:53.468408 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.468394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-552zh" Jul 06 23:42:53.468500 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.468484 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mm56r\"" Jul 06 23:42:53.470646 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.470614 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hmzwd" Jul 06 23:42:53.470997 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.470980 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-588bl\"" Jul 06 23:42:53.471401 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.471134 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 23:42:53.473999 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.473346 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 23:42:53.473999 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.473576 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.473999 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.473723 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 23:42:53.473999 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.473989 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:42:53.474768 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.474740 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hf4jw\"" Jul 06 23:42:53.475884 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.475859 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 23:42:53.476612 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.476566 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 23:42:53.476939 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.476919 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 23:42:53.477201 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.477177 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mj6sg\"" Jul 06 23:42:53.490645 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.490595 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pg745" Jul 06 23:42:53.498940 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.498871 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pg745" Jul 06 23:42:53.552606 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.552574 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 23:42:53.561893 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.561767 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-66.ec2.internal" event={"ID":"33725b36b540b2d810e88a2177f8bbbe","Type":"ContainerStarted","Data":"7ec3ca7dfb1bd515cccc57bf4f241ce55c0b6d43d1c5f81f2a501a066b38d070"} Jul 06 23:42:53.563150 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.563123 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" event={"ID":"891180d0e757f7b45d73ab1d4d4824ae","Type":"ContainerStarted","Data":"34122de97745f4db1fc6b5222624dd118405e6e9b6c2b6a007f0dd6dbdcee8c9"} Jul 06 23:42:53.564519 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564476 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f61dbcd-7f35-4161-8a71-2a737eed2ded-tmp-dir\") pod \"node-resolver-qcg78\" (UID: \"1f61dbcd-7f35-4161-8a71-2a737eed2ded\") " pod="openshift-dns/node-resolver-qcg78" Jul 06 23:42:53.564675 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564528 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-cni-netd\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.564675 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564559 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/753ab644-3d33-4875-bf8d-509d4451c8da-ovnkube-config\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.564675 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564608 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29gjk\" (UniqueName: \"kubernetes.io/projected/58cbc7f0-0b10-46b4-8f20-73034d089bc9-kube-api-access-29gjk\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:53.564675 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564639 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/466d19cf-93b7-43c7-81a6-7bbe860f4a99-cni-binary-copy\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.564675 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564661 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-hostroot\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.564954 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564687 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-node-log\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.564954 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564713 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/753ab644-3d33-4875-bf8d-509d4451c8da-ovnkube-script-lib\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.564954 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564741 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-tuned\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.564954 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564765 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c558h\" (UniqueName: \"kubernetes.io/projected/1f61dbcd-7f35-4161-8a71-2a737eed2ded-kube-api-access-c558h\") pod \"node-resolver-qcg78\" (UID: \"1f61dbcd-7f35-4161-8a71-2a737eed2ded\") " pod="openshift-dns/node-resolver-qcg78" Jul 06 23:42:53.564954 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564797 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd8zf\" (UniqueName: \"kubernetes.io/projected/454989b3-ca4f-4b8d-8dc0-17f00a0b55c3-kube-api-access-nd8zf\") pod \"node-ca-kbskp\" (UID: \"454989b3-ca4f-4b8d-8dc0-17f00a0b55c3\") " pod="openshift-image-registry/node-ca-kbskp" Jul 06 23:42:53.564954 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564821 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-var-lib-cni-multus\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.564954 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:53.564954 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564887 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/135de03b-04d2-40e8-b52e-d0f2ec945147-tmp\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.564954 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564914 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.564954 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564942 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-run-netns\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.564977 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a34c708d-ebcf-49e2-ad65-a18ece0d1858-konnectivity-ca\") pod \"konnectivity-agent-6l7np\" (UID: \"a34c708d-ebcf-49e2-ad65-a18ece0d1858\") " pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565006 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-var-lib-kubelet\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-etc-selinux\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565067 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-systemd\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565092 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/454989b3-ca4f-4b8d-8dc0-17f00a0b55c3-host\") pod \"node-ca-kbskp\" (UID: \"454989b3-ca4f-4b8d-8dc0-17f00a0b55c3\") " pod="openshift-image-registry/node-ca-kbskp" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565116 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-os-release\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-daemon-config\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-kubelet\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-os-release\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565250 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a524114-7358-43de-85d9-00e55e281c8a-dbus\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-sysconfig\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565313 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-run\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565350 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-host\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565376 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6dqh\" (UniqueName: \"kubernetes.io/projected/217e0df0-0e80-4f6e-82df-a4ca808f849c-kube-api-access-x6dqh\") pod \"iptables-alerter-hmzwd\" (UID: \"217e0df0-0e80-4f6e-82df-a4ca808f849c\") " pod="openshift-network-operator/iptables-alerter-hmzwd" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565408 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-system-cni-dir\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.565459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565449 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss7gx\" (UniqueName: \"kubernetes.io/projected/135de03b-04d2-40e8-b52e-d0f2ec945147-kube-api-access-ss7gx\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565481 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1f61dbcd-7f35-4161-8a71-2a737eed2ded-hosts-file\") pod \"node-resolver-qcg78\" (UID: \"1f61dbcd-7f35-4161-8a71-2a737eed2ded\") " pod="openshift-dns/node-resolver-qcg78" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565524 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-var-lib-cni-bin\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565555 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-registration-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zht6t\" (UniqueName: \"kubernetes.io/projected/6dfbb1c6-a6c0-4f03-8789-344efaece614-kube-api-access-zht6t\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565615 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/753ab644-3d33-4875-bf8d-509d4451c8da-ovn-node-metrics-cert\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565641 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-socket-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565690 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-device-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565725 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-systemd-units\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565754 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-cni-bin\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565782 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565813 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-modprobe-d\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565841 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-sysctl-d\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-slash\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565901 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-run-openvswitch\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565934 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-run-ovn-kubernetes\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.566286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.565972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-cnibin\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566003 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdtd\" (UniqueName: \"kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd\") pod \"network-check-target-vlbd8\" (UID: \"d5222093-6d7d-46d1-93f1-47245a02ae3c\") " pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566068 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-sysctl-conf\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566097 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-lib-modules\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566129 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-run-systemd\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566158 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-run-ovn\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566187 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2db27634-076c-484d-a1bf-9633194d39c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566219 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a524114-7358-43de-85d9-00e55e281c8a-kubelet-config\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-sys\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566275 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-cnibin\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566303 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-run-k8s-cni-cncf-io\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566333 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-sys-fs\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566362 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-var-lib-openvswitch\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566394 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-system-cni-dir\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566421 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-cni-dir\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566448 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/217e0df0-0e80-4f6e-82df-a4ca808f849c-host-slash\") pod \"iptables-alerter-hmzwd\" (UID: \"217e0df0-0e80-4f6e-82df-a4ca808f849c\") " pod="openshift-network-operator/iptables-alerter-hmzwd" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566478 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2p9\" (UniqueName: \"kubernetes.io/projected/753ab644-3d33-4875-bf8d-509d4451c8da-kube-api-access-gb2p9\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.567117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2db27634-076c-484d-a1bf-9633194d39c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566567 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-socket-dir-parent\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566594 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-conf-dir\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566627 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a34c708d-ebcf-49e2-ad65-a18ece0d1858-agent-certs\") pod \"konnectivity-agent-6l7np\" (UID: \"a34c708d-ebcf-49e2-ad65-a18ece0d1858\") " pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566676 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566705 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-var-lib-kubelet\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566734 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-run-netns\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566766 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-run-multus-certs\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566794 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml4xp\" (UniqueName: \"kubernetes.io/projected/466d19cf-93b7-43c7-81a6-7bbe860f4a99-kube-api-access-ml4xp\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566854 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-etc-openvswitch\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-log-socket\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566917 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2db27634-076c-484d-a1bf-9633194d39c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566947 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kcjv\" (UniqueName: \"kubernetes.io/projected/2db27634-076c-484d-a1bf-9633194d39c6-kube-api-access-9kcjv\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.566978 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/454989b3-ca4f-4b8d-8dc0-17f00a0b55c3-serviceca\") pod \"node-ca-kbskp\" (UID: \"454989b3-ca4f-4b8d-8dc0-17f00a0b55c3\") " pod="openshift-image-registry/node-ca-kbskp" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.567010 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-etc-kubernetes\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.567040 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/217e0df0-0e80-4f6e-82df-a4ca808f849c-iptables-alerter-script\") pod \"iptables-alerter-hmzwd\" (UID: \"217e0df0-0e80-4f6e-82df-a4ca808f849c\") " pod="openshift-network-operator/iptables-alerter-hmzwd" Jul 06 23:42:53.567964 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.567073 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/753ab644-3d33-4875-bf8d-509d4451c8da-env-overrides\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.568813 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.567102 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-kubernetes\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.568813 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.567236 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-cni-netd\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.568813 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.567962 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/753ab644-3d33-4875-bf8d-509d4451c8da-ovnkube-config\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.568813 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.568383 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-node-log\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.569029 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.568977 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/753ab644-3d33-4875-bf8d-509d4451c8da-ovnkube-script-lib\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.569148 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:53.569127 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:42:53.569227 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:53.569213 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret podName:2a524114-7358-43de-85d9-00e55e281c8a nodeName:}" failed. No retries permitted until 2026-07-06 23:42:54.069182613 +0000 UTC m=+2.961371285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret") pod "global-pull-secret-syncer-nng68" (UID: "2a524114-7358-43de-85d9-00e55e281c8a") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:42:53.569506 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.569457 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-run-netns\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.569652 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.569603 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-run-systemd\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.569725 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.569706 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-kubelet\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.569843 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.569815 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-run-ovn-kubernetes\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.569914 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:53.569851 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:42:53.569914 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.569878 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 23:42:53.569914 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:53.569909 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs podName:58cbc7f0-0b10-46b4-8f20-73034d089bc9 nodeName:}" failed. No retries permitted until 2026-07-06 23:42:54.069888891 +0000 UTC m=+2.962077570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs") pod "network-metrics-daemon-f2vgw" (UID: "58cbc7f0-0b10-46b4-8f20-73034d089bc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:42:53.569914 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.569905 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-cnibin\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.570118 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.569952 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-log-socket\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.570118 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.569980 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a524114-7358-43de-85d9-00e55e281c8a-dbus\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:53.570118 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570032 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-systemd-units\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.570118 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570067 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-system-cni-dir\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.570348 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570330 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-cni-bin\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.570449 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570364 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-run-ovn\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.570449 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570389 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.570449 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570431 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2db27634-076c-484d-a1bf-9633194d39c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.570611 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570552 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.570611 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570572 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a34c708d-ebcf-49e2-ad65-a18ece0d1858-konnectivity-ca\") pod \"konnectivity-agent-6l7np\" (UID: \"a34c708d-ebcf-49e2-ad65-a18ece0d1858\") " pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:42:53.570611 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570594 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-etc-openvswitch\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.570796 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570682 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-run-openvswitch\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.570796 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570720 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-host-slash\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.570796 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570742 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a524114-7358-43de-85d9-00e55e281c8a-kubelet-config\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:53.570975 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570799 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/753ab644-3d33-4875-bf8d-509d4451c8da-var-lib-openvswitch\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.570975 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570844 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2db27634-076c-484d-a1bf-9633194d39c6-os-release\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.570975 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.570868 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2db27634-076c-484d-a1bf-9633194d39c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.571168 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.571138 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2db27634-076c-484d-a1bf-9633194d39c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.571827 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.571782 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/753ab644-3d33-4875-bf8d-509d4451c8da-env-overrides\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.574689 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.574293 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/753ab644-3d33-4875-bf8d-509d4451c8da-ovn-node-metrics-cert\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.574689 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.574534 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a34c708d-ebcf-49e2-ad65-a18ece0d1858-agent-certs\") pod \"konnectivity-agent-6l7np\" (UID: \"a34c708d-ebcf-49e2-ad65-a18ece0d1858\") " pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:42:53.578795 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:53.578748 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:42:53.578795 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:53.578783 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:42:53.578795 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:53.578797 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmdtd for pod openshift-network-diagnostics/network-check-target-vlbd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:42:53.579121 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:53.578876 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd podName:d5222093-6d7d-46d1-93f1-47245a02ae3c nodeName:}" failed. No retries permitted until 2026-07-06 23:42:54.078850507 +0000 UTC m=+2.971039180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vmdtd" (UniqueName: "kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd") pod "network-check-target-vlbd8" (UID: "d5222093-6d7d-46d1-93f1-47245a02ae3c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:42:53.581598 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.581561 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29gjk\" (UniqueName: \"kubernetes.io/projected/58cbc7f0-0b10-46b4-8f20-73034d089bc9-kube-api-access-29gjk\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:53.582062 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.582032 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kcjv\" (UniqueName: \"kubernetes.io/projected/2db27634-076c-484d-a1bf-9633194d39c6-kube-api-access-9kcjv\") pod \"multus-additional-cni-plugins-tk9hj\" (UID: \"2db27634-076c-484d-a1bf-9633194d39c6\") " pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.583412 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.583385 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb2p9\" (UniqueName: \"kubernetes.io/projected/753ab644-3d33-4875-bf8d-509d4451c8da-kube-api-access-gb2p9\") pod \"ovnkube-node-92dbw\" (UID: \"753ab644-3d33-4875-bf8d-509d4451c8da\") " pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.667740 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.667694 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-etc-selinux\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.667740 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.667744 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-systemd\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.668032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.667770 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/454989b3-ca4f-4b8d-8dc0-17f00a0b55c3-host\") pod \"node-ca-kbskp\" (UID: \"454989b3-ca4f-4b8d-8dc0-17f00a0b55c3\") " pod="openshift-image-registry/node-ca-kbskp" Jul 06 23:42:53.668032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.667794 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-os-release\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.668032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.667813 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-daemon-config\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.668032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.667843 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-sysconfig\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.668032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.667852 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-systemd\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.668032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.667900 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-etc-selinux\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.668032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.667933 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/454989b3-ca4f-4b8d-8dc0-17f00a0b55c3-host\") pod \"node-ca-kbskp\" (UID: \"454989b3-ca4f-4b8d-8dc0-17f00a0b55c3\") " pod="openshift-image-registry/node-ca-kbskp" Jul 06 23:42:53.668032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.667940 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-os-release\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.668032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.667989 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-run\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.668032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668020 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-host\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.668032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668032 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-sysconfig\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668047 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6dqh\" (UniqueName: \"kubernetes.io/projected/217e0df0-0e80-4f6e-82df-a4ca808f849c-kube-api-access-x6dqh\") pod \"iptables-alerter-hmzwd\" (UID: \"217e0df0-0e80-4f6e-82df-a4ca808f849c\") " pod="openshift-network-operator/iptables-alerter-hmzwd" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668078 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-host\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668081 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss7gx\" (UniqueName: \"kubernetes.io/projected/135de03b-04d2-40e8-b52e-d0f2ec945147-kube-api-access-ss7gx\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668113 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1f61dbcd-7f35-4161-8a71-2a737eed2ded-hosts-file\") pod \"node-resolver-qcg78\" (UID: \"1f61dbcd-7f35-4161-8a71-2a737eed2ded\") " pod="openshift-dns/node-resolver-qcg78" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668051 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-run\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668142 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-var-lib-cni-bin\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668173 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-registration-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zht6t\" (UniqueName: \"kubernetes.io/projected/6dfbb1c6-a6c0-4f03-8789-344efaece614-kube-api-access-zht6t\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-socket-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-device-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-modprobe-d\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668317 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-sysctl-d\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668335 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-registration-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668357 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-daemon-config\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668399 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1f61dbcd-7f35-4161-8a71-2a737eed2ded-hosts-file\") pod \"node-resolver-qcg78\" (UID: \"1f61dbcd-7f35-4161-8a71-2a737eed2ded\") " pod="openshift-dns/node-resolver-qcg78" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668415 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-device-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.668573 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668433 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-sysctl-conf\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668446 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-var-lib-cni-bin\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668467 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-lib-modules\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668487 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-sys\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-cnibin\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-run-k8s-cni-cncf-io\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668533 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-sysctl-d\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668536 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-sys-fs\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668569 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-sys-fs\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-system-cni-dir\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668612 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-socket-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668633 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-cni-dir\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668662 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/217e0df0-0e80-4f6e-82df-a4ca808f849c-host-slash\") pod \"iptables-alerter-hmzwd\" (UID: \"217e0df0-0e80-4f6e-82df-a4ca808f849c\") " pod="openshift-network-operator/iptables-alerter-hmzwd" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668665 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-sys\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668699 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-sysctl-conf\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-socket-dir-parent\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668746 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-conf-dir\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668768 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-cnibin\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.669484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-var-lib-kubelet\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668804 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-run-k8s-cni-cncf-io\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668523 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-modprobe-d\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668820 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-lib-modules\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668854 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/217e0df0-0e80-4f6e-82df-a4ca808f849c-host-slash\") pod \"iptables-alerter-hmzwd\" (UID: \"217e0df0-0e80-4f6e-82df-a4ca808f849c\") " pod="openshift-network-operator/iptables-alerter-hmzwd" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668870 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-cni-dir\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668875 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-conf-dir\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668883 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-multus-socket-dir-parent\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668911 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-run-netns\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668945 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-run-multus-certs\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668916 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-system-cni-dir\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668973 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml4xp\" (UniqueName: \"kubernetes.io/projected/466d19cf-93b7-43c7-81a6-7bbe860f4a99-kube-api-access-ml4xp\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.668904 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-var-lib-kubelet\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669007 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-run-netns\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669006 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/454989b3-ca4f-4b8d-8dc0-17f00a0b55c3-serviceca\") pod \"node-ca-kbskp\" (UID: \"454989b3-ca4f-4b8d-8dc0-17f00a0b55c3\") " pod="openshift-image-registry/node-ca-kbskp" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669027 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-run-multus-certs\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669059 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-etc-kubernetes\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669093 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/217e0df0-0e80-4f6e-82df-a4ca808f849c-iptables-alerter-script\") pod \"iptables-alerter-hmzwd\" (UID: \"217e0df0-0e80-4f6e-82df-a4ca808f849c\") " pod="openshift-network-operator/iptables-alerter-hmzwd" Jul 06 23:42:53.670595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669179 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-etc-kubernetes\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-kubernetes\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669216 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f61dbcd-7f35-4161-8a71-2a737eed2ded-tmp-dir\") pod \"node-resolver-qcg78\" (UID: \"1f61dbcd-7f35-4161-8a71-2a737eed2ded\") " pod="openshift-dns/node-resolver-qcg78" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669290 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/466d19cf-93b7-43c7-81a6-7bbe860f4a99-cni-binary-copy\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-hostroot\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669300 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-kubernetes\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669382 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-tuned\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669401 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c558h\" (UniqueName: \"kubernetes.io/projected/1f61dbcd-7f35-4161-8a71-2a737eed2ded-kube-api-access-c558h\") pod \"node-resolver-qcg78\" (UID: \"1f61dbcd-7f35-4161-8a71-2a737eed2ded\") " pod="openshift-dns/node-resolver-qcg78" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669431 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd8zf\" (UniqueName: \"kubernetes.io/projected/454989b3-ca4f-4b8d-8dc0-17f00a0b55c3-kube-api-access-nd8zf\") pod \"node-ca-kbskp\" (UID: \"454989b3-ca4f-4b8d-8dc0-17f00a0b55c3\") " pod="openshift-image-registry/node-ca-kbskp" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669454 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-var-lib-cni-multus\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669535 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/135de03b-04d2-40e8-b52e-d0f2ec945147-tmp\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669571 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669581 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/454989b3-ca4f-4b8d-8dc0-17f00a0b55c3-serviceca\") pod \"node-ca-kbskp\" (UID: \"454989b3-ca4f-4b8d-8dc0-17f00a0b55c3\") " pod="openshift-image-registry/node-ca-kbskp" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669596 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-var-lib-kubelet\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669713 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-var-lib-cni-multus\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669888 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/217e0df0-0e80-4f6e-82df-a4ca808f849c-iptables-alerter-script\") pod \"iptables-alerter-hmzwd\" (UID: \"217e0df0-0e80-4f6e-82df-a4ca808f849c\") " pod="openshift-network-operator/iptables-alerter-hmzwd" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669916 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/466d19cf-93b7-43c7-81a6-7bbe860f4a99-cni-binary-copy\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.669953 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-host-var-lib-kubelet\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.671345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.670013 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6dfbb1c6-a6c0-4f03-8789-344efaece614-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.672120 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.670296 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f61dbcd-7f35-4161-8a71-2a737eed2ded-tmp-dir\") pod \"node-resolver-qcg78\" (UID: \"1f61dbcd-7f35-4161-8a71-2a737eed2ded\") " pod="openshift-dns/node-resolver-qcg78" Jul 06 23:42:53.672120 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.670299 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/466d19cf-93b7-43c7-81a6-7bbe860f4a99-hostroot\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.672963 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.672926 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/135de03b-04d2-40e8-b52e-d0f2ec945147-tmp\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.673088 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.672974 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/135de03b-04d2-40e8-b52e-d0f2ec945147-etc-tuned\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.693037 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.692972 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss7gx\" (UniqueName: \"kubernetes.io/projected/135de03b-04d2-40e8-b52e-d0f2ec945147-kube-api-access-ss7gx\") pod \"tuned-zjxbk\" (UID: \"135de03b-04d2-40e8-b52e-d0f2ec945147\") " pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.693231 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.693059 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6dqh\" (UniqueName: \"kubernetes.io/projected/217e0df0-0e80-4f6e-82df-a4ca808f849c-kube-api-access-x6dqh\") pod \"iptables-alerter-hmzwd\" (UID: \"217e0df0-0e80-4f6e-82df-a4ca808f849c\") " pod="openshift-network-operator/iptables-alerter-hmzwd" Jul 06 23:42:53.693231 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.693122 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zht6t\" (UniqueName: \"kubernetes.io/projected/6dfbb1c6-a6c0-4f03-8789-344efaece614-kube-api-access-zht6t\") pod \"aws-ebs-csi-driver-node-8j5pg\" (UID: \"6dfbb1c6-a6c0-4f03-8789-344efaece614\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:53.693231 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.693148 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c558h\" (UniqueName: \"kubernetes.io/projected/1f61dbcd-7f35-4161-8a71-2a737eed2ded-kube-api-access-c558h\") pod \"node-resolver-qcg78\" (UID: \"1f61dbcd-7f35-4161-8a71-2a737eed2ded\") " pod="openshift-dns/node-resolver-qcg78" Jul 06 23:42:53.693911 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.693877 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd8zf\" (UniqueName: \"kubernetes.io/projected/454989b3-ca4f-4b8d-8dc0-17f00a0b55c3-kube-api-access-nd8zf\") pod \"node-ca-kbskp\" (UID: \"454989b3-ca4f-4b8d-8dc0-17f00a0b55c3\") " pod="openshift-image-registry/node-ca-kbskp" Jul 06 23:42:53.695279 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.695252 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml4xp\" (UniqueName: \"kubernetes.io/projected/466d19cf-93b7-43c7-81a6-7bbe860f4a99-kube-api-access-ml4xp\") pod \"multus-552zh\" (UID: \"466d19cf-93b7-43c7-81a6-7bbe860f4a99\") " pod="openshift-multus/multus-552zh" Jul 06 23:42:53.764494 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.764415 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:42:53.775169 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.775137 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tk9hj" Jul 06 23:42:53.793349 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.793304 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:42:53.798826 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.798784 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" Jul 06 23:42:53.809007 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.808967 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qcg78" Jul 06 23:42:53.816442 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.816412 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kbskp" Jul 06 23:42:53.825844 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.825805 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-552zh" Jul 06 23:42:53.836099 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.836067 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hmzwd" Jul 06 23:42:53.844415 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:53.844377 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" Jul 06 23:42:54.073640 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:54.073514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:54.073640 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:54.073571 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:54.073884 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:54.073701 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:42:54.073884 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:54.073713 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:42:54.073884 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:54.073782 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret podName:2a524114-7358-43de-85d9-00e55e281c8a nodeName:}" failed. No retries permitted until 2026-07-06 23:42:55.073761773 +0000 UTC m=+3.965950431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret") pod "global-pull-secret-syncer-nng68" (UID: "2a524114-7358-43de-85d9-00e55e281c8a") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:42:54.073884 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:54.073798 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs podName:58cbc7f0-0b10-46b4-8f20-73034d089bc9 nodeName:}" failed. No retries permitted until 2026-07-06 23:42:55.07379052 +0000 UTC m=+3.965979177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs") pod "network-metrics-daemon-f2vgw" (UID: "58cbc7f0-0b10-46b4-8f20-73034d089bc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:42:54.174013 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:54.173964 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdtd\" (UniqueName: \"kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd\") pod \"network-check-target-vlbd8\" (UID: \"d5222093-6d7d-46d1-93f1-47245a02ae3c\") " pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:42:54.174242 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:54.174196 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:42:54.174242 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:54.174225 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:42:54.174242 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:54.174241 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmdtd for pod openshift-network-diagnostics/network-check-target-vlbd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:42:54.174390 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:54.174317 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd podName:d5222093-6d7d-46d1-93f1-47245a02ae3c nodeName:}" failed. No retries permitted until 2026-07-06 23:42:55.174292767 +0000 UTC m=+4.066481440 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmdtd" (UniqueName: "kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd") pod "network-check-target-vlbd8" (UID: "d5222093-6d7d-46d1-93f1-47245a02ae3c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:42:54.502652 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:54.502575 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:37:53 +0000 UTC" deadline="2028-04-21 12:12:34.078529828 +0000 UTC" Jul 06 23:42:54.502652 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:54.502643 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15708h29m39.575891978s" Jul 06 23:42:54.557154 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:54.557117 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:54.557365 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:54.557258 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:42:54.697906 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:54.697868 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod454989b3_ca4f_4b8d_8dc0_17f00a0b55c3.slice/crio-d0ebdf108dec6b2e6cb8cbc97d3d30a73aa7aa099c03290590deed46eb847b85 WatchSource:0}: Error finding container d0ebdf108dec6b2e6cb8cbc97d3d30a73aa7aa099c03290590deed46eb847b85: Status 404 returned error can't find the container with id d0ebdf108dec6b2e6cb8cbc97d3d30a73aa7aa099c03290590deed46eb847b85 Jul 06 23:42:54.699615 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:54.699586 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34c708d_ebcf_49e2_ad65_a18ece0d1858.slice/crio-0cfa7e7e1444c3622c00c8cf170039f7116f3fdc6fecff2352c24ba06dd00843 WatchSource:0}: Error finding container 0cfa7e7e1444c3622c00c8cf170039f7116f3fdc6fecff2352c24ba06dd00843: Status 404 returned error can't find the container with id 0cfa7e7e1444c3622c00c8cf170039f7116f3fdc6fecff2352c24ba06dd00843 Jul 06 23:42:54.701364 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:54.701334 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod135de03b_04d2_40e8_b52e_d0f2ec945147.slice/crio-1aad608985c7a8a1827d7f35ceeb50408ccfb59bb0aa462d6e49cf09fed9e233 WatchSource:0}: Error finding container 1aad608985c7a8a1827d7f35ceeb50408ccfb59bb0aa462d6e49cf09fed9e233: Status 404 returned error can't find the container with id 1aad608985c7a8a1827d7f35ceeb50408ccfb59bb0aa462d6e49cf09fed9e233 Jul 06 23:42:54.707864 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:54.707825 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dfbb1c6_a6c0_4f03_8789_344efaece614.slice/crio-75f01bb0efb849415bd7d035835909d7ca5af61e23ac2640cc91d4fb621bc910 WatchSource:0}: Error finding container 75f01bb0efb849415bd7d035835909d7ca5af61e23ac2640cc91d4fb621bc910: Status 404 returned error can't find the container with id 75f01bb0efb849415bd7d035835909d7ca5af61e23ac2640cc91d4fb621bc910 Jul 06 23:42:54.710499 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:54.710438 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod217e0df0_0e80_4f6e_82df_a4ca808f849c.slice/crio-b8ef7687751b923228690556bcabf486ca2306c4118594f1376f592956f376d1 WatchSource:0}: Error finding container b8ef7687751b923228690556bcabf486ca2306c4118594f1376f592956f376d1: Status 404 returned error can't find the container with id b8ef7687751b923228690556bcabf486ca2306c4118594f1376f592956f376d1 Jul 06 23:42:54.712071 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:54.712048 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod466d19cf_93b7_43c7_81a6_7bbe860f4a99.slice/crio-f753ff49bedc21e2b036eac8ec13a49b06c5fb0de92779edecac4780b3987f4d WatchSource:0}: Error finding container f753ff49bedc21e2b036eac8ec13a49b06c5fb0de92779edecac4780b3987f4d: Status 404 returned error can't find the container with id f753ff49bedc21e2b036eac8ec13a49b06c5fb0de92779edecac4780b3987f4d Jul 06 23:42:54.714422 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:54.713960 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod753ab644_3d33_4875_bf8d_509d4451c8da.slice/crio-6bc7bb982039928a3ff0f92965740be0e5309850a404238c7680f036e1d4a2c0 WatchSource:0}: Error finding container 6bc7bb982039928a3ff0f92965740be0e5309850a404238c7680f036e1d4a2c0: Status 404 returned error can't find the container with id 6bc7bb982039928a3ff0f92965740be0e5309850a404238c7680f036e1d4a2c0 Jul 06 23:42:54.715299 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:42:54.715262 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f61dbcd_7f35_4161_8a71_2a737eed2ded.slice/crio-ac17fbf80ad288b36419a2e54b2f301580cee1423f6f52e936e3c77b1b43ca8b WatchSource:0}: Error finding container ac17fbf80ad288b36419a2e54b2f301580cee1423f6f52e936e3c77b1b43ca8b: Status 404 returned error can't find the container with id ac17fbf80ad288b36419a2e54b2f301580cee1423f6f52e936e3c77b1b43ca8b Jul 06 23:42:55.082180 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.081902 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:55.082469 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:55.082102 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:42:55.082469 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.082259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:55.082469 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:55.082308 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret podName:2a524114-7358-43de-85d9-00e55e281c8a nodeName:}" failed. No retries permitted until 2026-07-06 23:42:57.082281721 +0000 UTC m=+5.974470401 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret") pod "global-pull-secret-syncer-nng68" (UID: "2a524114-7358-43de-85d9-00e55e281c8a") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:42:55.082469 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:55.082395 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:42:55.082469 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:55.082460 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs podName:58cbc7f0-0b10-46b4-8f20-73034d089bc9 nodeName:}" failed. No retries permitted until 2026-07-06 23:42:57.082441603 +0000 UTC m=+5.974630260 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs") pod "network-metrics-daemon-f2vgw" (UID: "58cbc7f0-0b10-46b4-8f20-73034d089bc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:42:55.183107 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.183056 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdtd\" (UniqueName: \"kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd\") pod \"network-check-target-vlbd8\" (UID: \"d5222093-6d7d-46d1-93f1-47245a02ae3c\") " pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:42:55.183329 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:55.183255 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:42:55.183329 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:55.183278 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:42:55.183329 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:55.183294 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmdtd for pod openshift-network-diagnostics/network-check-target-vlbd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:42:55.183508 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:55.183364 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd podName:d5222093-6d7d-46d1-93f1-47245a02ae3c nodeName:}" failed. No retries permitted until 2026-07-06 23:42:57.183342149 +0000 UTC m=+6.075530827 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmdtd" (UniqueName: "kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd") pod "network-check-target-vlbd8" (UID: "d5222093-6d7d-46d1-93f1-47245a02ae3c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:42:55.503411 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.503350 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:37:53 +0000 UTC" deadline="2028-03-17 04:01:03.028984844 +0000 UTC" Jul 06 23:42:55.503411 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.503397 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14860h18m7.525592678s" Jul 06 23:42:55.560865 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.560059 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:55.560865 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:55.560238 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:42:55.561127 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.561046 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:42:55.561189 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:55.561173 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:42:55.581716 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.581620 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" event={"ID":"135de03b-04d2-40e8-b52e-d0f2ec945147","Type":"ContainerStarted","Data":"1aad608985c7a8a1827d7f35ceeb50408ccfb59bb0aa462d6e49cf09fed9e233"} Jul 06 23:42:55.590796 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.590714 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kbskp" event={"ID":"454989b3-ca4f-4b8d-8dc0-17f00a0b55c3","Type":"ContainerStarted","Data":"d0ebdf108dec6b2e6cb8cbc97d3d30a73aa7aa099c03290590deed46eb847b85"} Jul 06 23:42:55.606012 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.605961 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qcg78" event={"ID":"1f61dbcd-7f35-4161-8a71-2a737eed2ded","Type":"ContainerStarted","Data":"ac17fbf80ad288b36419a2e54b2f301580cee1423f6f52e936e3c77b1b43ca8b"} Jul 06 23:42:55.612812 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.612762 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" event={"ID":"753ab644-3d33-4875-bf8d-509d4451c8da","Type":"ContainerStarted","Data":"6bc7bb982039928a3ff0f92965740be0e5309850a404238c7680f036e1d4a2c0"} Jul 06 23:42:55.615465 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.615414 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" event={"ID":"6dfbb1c6-a6c0-4f03-8789-344efaece614","Type":"ContainerStarted","Data":"75f01bb0efb849415bd7d035835909d7ca5af61e23ac2640cc91d4fb621bc910"} Jul 06 23:42:55.626004 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.625954 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6l7np" event={"ID":"a34c708d-ebcf-49e2-ad65-a18ece0d1858","Type":"ContainerStarted","Data":"0cfa7e7e1444c3622c00c8cf170039f7116f3fdc6fecff2352c24ba06dd00843"} Jul 06 23:42:55.647535 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.647490 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-66.ec2.internal" event={"ID":"33725b36b540b2d810e88a2177f8bbbe","Type":"ContainerStarted","Data":"30bb6fa08ec6bcaa63281f74b323c56ef59877eb7860a06b24aec398d5632ea5"} Jul 06 23:42:55.657299 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.657247 2560 generic.go:356] "Generic (PLEG): container finished" podID="891180d0e757f7b45d73ab1d4d4824ae" containerID="5ec8fe39c710a5e6341682bdec8966a43be790f7eafb41448a70826c9a93d3b6" exitCode=0 Jul 06 23:42:55.657533 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.657391 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" event={"ID":"891180d0e757f7b45d73ab1d4d4824ae","Type":"ContainerDied","Data":"5ec8fe39c710a5e6341682bdec8966a43be790f7eafb41448a70826c9a93d3b6"} Jul 06 23:42:55.663505 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.663421 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-66.ec2.internal" podStartSLOduration=2.663396991 podStartE2EDuration="2.663396991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:42:55.663200686 +0000 UTC m=+4.555389370" watchObservedRunningTime="2026-07-06 23:42:55.663396991 +0000 UTC m=+4.555585674" Jul 06 23:42:55.666433 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.666393 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tk9hj" event={"ID":"2db27634-076c-484d-a1bf-9633194d39c6","Type":"ContainerStarted","Data":"2dd371df6a74758bc8e3b48807fd4206aa74eb25386a2fa53ce1bea0dde27851"} Jul 06 23:42:55.682275 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.682224 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-552zh" event={"ID":"466d19cf-93b7-43c7-81a6-7bbe860f4a99","Type":"ContainerStarted","Data":"f753ff49bedc21e2b036eac8ec13a49b06c5fb0de92779edecac4780b3987f4d"} Jul 06 23:42:55.696001 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:55.695955 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hmzwd" event={"ID":"217e0df0-0e80-4f6e-82df-a4ca808f849c","Type":"ContainerStarted","Data":"b8ef7687751b923228690556bcabf486ca2306c4118594f1376f592956f376d1"} Jul 06 23:42:56.557280 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:56.557236 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:56.557815 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:56.557409 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:42:56.725492 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:56.725447 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" event={"ID":"891180d0e757f7b45d73ab1d4d4824ae","Type":"ContainerStarted","Data":"91c69377fdbeb5bed724999f1484c31840bfbaaa2e4c1ab03c2d18bca1094677"} Jul 06 23:42:57.100583 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:57.100521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:57.100791 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:57.100647 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:57.100854 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:57.100793 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:42:57.100904 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:57.100860 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs podName:58cbc7f0-0b10-46b4-8f20-73034d089bc9 nodeName:}" failed. No retries permitted until 2026-07-06 23:43:01.100840747 +0000 UTC m=+9.993029414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs") pod "network-metrics-daemon-f2vgw" (UID: "58cbc7f0-0b10-46b4-8f20-73034d089bc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:42:57.101295 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:57.101272 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:42:57.101370 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:57.101340 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret podName:2a524114-7358-43de-85d9-00e55e281c8a nodeName:}" failed. No retries permitted until 2026-07-06 23:43:01.101322477 +0000 UTC m=+9.993511137 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret") pod "global-pull-secret-syncer-nng68" (UID: "2a524114-7358-43de-85d9-00e55e281c8a") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:42:57.201231 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:57.201185 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdtd\" (UniqueName: \"kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd\") pod \"network-check-target-vlbd8\" (UID: \"d5222093-6d7d-46d1-93f1-47245a02ae3c\") " pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:42:57.201518 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:57.201357 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:42:57.201518 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:57.201380 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:42:57.201518 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:57.201396 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmdtd for pod openshift-network-diagnostics/network-check-target-vlbd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:42:57.201518 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:57.201471 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd podName:d5222093-6d7d-46d1-93f1-47245a02ae3c nodeName:}" failed. No retries permitted until 2026-07-06 23:43:01.201445132 +0000 UTC m=+10.093633791 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmdtd" (UniqueName: "kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd") pod "network-check-target-vlbd8" (UID: "d5222093-6d7d-46d1-93f1-47245a02ae3c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:42:57.557857 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:57.557762 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:42:57.558348 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:57.557910 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:42:57.560509 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:57.560483 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:57.560686 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:57.560653 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:42:58.557081 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:58.557046 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:42:58.557262 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:58.557192 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:42:59.559238 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:59.558686 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:42:59.559238 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:59.558842 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:42:59.560104 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:42:59.559934 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:42:59.560104 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:42:59.560056 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:00.557177 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:00.557137 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:00.557370 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:00.557313 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:01.133938 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:01.133900 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:01.134376 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:01.133971 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:01.134376 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:01.134115 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:43:01.134376 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:01.134130 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:43:01.134376 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:01.134180 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs podName:58cbc7f0-0b10-46b4-8f20-73034d089bc9 nodeName:}" failed. No retries permitted until 2026-07-06 23:43:09.134165036 +0000 UTC m=+18.026353695 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs") pod "network-metrics-daemon-f2vgw" (UID: "58cbc7f0-0b10-46b4-8f20-73034d089bc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:43:01.134376 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:01.134201 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret podName:2a524114-7358-43de-85d9-00e55e281c8a nodeName:}" failed. No retries permitted until 2026-07-06 23:43:09.134191029 +0000 UTC m=+18.026379686 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret") pod "global-pull-secret-syncer-nng68" (UID: "2a524114-7358-43de-85d9-00e55e281c8a") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:43:01.235319 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:01.235276 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdtd\" (UniqueName: \"kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd\") pod \"network-check-target-vlbd8\" (UID: \"d5222093-6d7d-46d1-93f1-47245a02ae3c\") " pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:01.235504 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:01.235446 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:43:01.235504 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:01.235462 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:43:01.235504 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:01.235471 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmdtd for pod openshift-network-diagnostics/network-check-target-vlbd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:43:01.235670 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:01.235516 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd podName:d5222093-6d7d-46d1-93f1-47245a02ae3c nodeName:}" failed. No retries permitted until 2026-07-06 23:43:09.235502949 +0000 UTC m=+18.127691606 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmdtd" (UniqueName: "kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd") pod "network-check-target-vlbd8" (UID: "d5222093-6d7d-46d1-93f1-47245a02ae3c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:43:01.558058 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:01.557972 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:01.558257 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:01.558078 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:01.558648 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:01.558495 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:01.558648 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:01.558609 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:02.557697 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:02.557404 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:02.557697 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:02.557648 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:03.557353 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:03.557318 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:03.557586 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:03.557451 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:03.558050 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:03.557904 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:03.558050 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:03.558011 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:04.557606 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:04.557547 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:04.557848 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:04.557724 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:05.557327 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:05.557284 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:05.557327 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:05.557311 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:05.557806 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:05.557413 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:05.557806 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:05.557526 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:06.557341 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:06.557302 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:06.557809 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:06.557435 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:07.557129 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:07.556956 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:07.557129 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:07.556965 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:07.557396 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:07.557164 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:07.557396 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:07.557286 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:08.557432 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:08.557389 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:08.558008 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:08.557522 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:09.201675 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:09.201627 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:09.201894 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:09.201735 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:09.201894 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:09.201809 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:43:09.201894 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:09.201883 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:43:09.202058 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:09.201894 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret podName:2a524114-7358-43de-85d9-00e55e281c8a nodeName:}" failed. No retries permitted until 2026-07-06 23:43:25.20186942 +0000 UTC m=+34.094058090 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret") pod "global-pull-secret-syncer-nng68" (UID: "2a524114-7358-43de-85d9-00e55e281c8a") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:43:09.202058 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:09.201958 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs podName:58cbc7f0-0b10-46b4-8f20-73034d089bc9 nodeName:}" failed. No retries permitted until 2026-07-06 23:43:25.201937747 +0000 UTC m=+34.094126413 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs") pod "network-metrics-daemon-f2vgw" (UID: "58cbc7f0-0b10-46b4-8f20-73034d089bc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:43:09.302500 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:09.302454 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdtd\" (UniqueName: \"kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd\") pod \"network-check-target-vlbd8\" (UID: \"d5222093-6d7d-46d1-93f1-47245a02ae3c\") " pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:09.302689 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:09.302644 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:43:09.302689 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:09.302668 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:43:09.302689 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:09.302683 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmdtd for pod openshift-network-diagnostics/network-check-target-vlbd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:43:09.302826 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:09.302756 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd podName:d5222093-6d7d-46d1-93f1-47245a02ae3c nodeName:}" failed. No retries permitted until 2026-07-06 23:43:25.302735952 +0000 UTC m=+34.194924623 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmdtd" (UniqueName: "kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd") pod "network-check-target-vlbd8" (UID: "d5222093-6d7d-46d1-93f1-47245a02ae3c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:43:09.557232 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:09.557140 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:09.557361 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:09.557268 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:09.557361 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:09.557329 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:09.557462 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:09.557440 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:10.556859 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:10.556824 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:10.557041 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:10.556963 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:11.558761 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:11.558719 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:11.559171 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:11.558856 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:11.559221 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:11.559190 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:11.559298 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:11.559264 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:12.557590 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:12.557529 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:12.557806 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:12.557706 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:13.556888 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:13.556834 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:13.557633 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:13.557034 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:13.557633 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:13.557096 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:13.557633 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:13.557210 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:14.557674 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.557299 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:14.558364 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:14.557774 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:14.769090 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.769054 2560 generic.go:356] "Generic (PLEG): container finished" podID="2db27634-076c-484d-a1bf-9633194d39c6" containerID="a8861541af1ca1290d2d7b9e02973d40997ea1a96cdbedc46e3ef69af4f9f930" exitCode=0 Jul 06 23:43:14.769293 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.769156 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tk9hj" event={"ID":"2db27634-076c-484d-a1bf-9633194d39c6","Type":"ContainerDied","Data":"a8861541af1ca1290d2d7b9e02973d40997ea1a96cdbedc46e3ef69af4f9f930"} Jul 06 23:43:14.771419 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.771389 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-552zh" event={"ID":"466d19cf-93b7-43c7-81a6-7bbe860f4a99","Type":"ContainerStarted","Data":"242fdc94c33ff0f3f3d64c0c5f334f84d95771d6a6bf10c5d23be67b2e8cae40"} Jul 06 23:43:14.774733 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.774411 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" event={"ID":"135de03b-04d2-40e8-b52e-d0f2ec945147","Type":"ContainerStarted","Data":"6bf3bb89cdebd1b804cfc96bbe4d2708caf45f503c2412fdd4daeb323c77307a"} Jul 06 23:43:14.781520 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.778762 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kbskp" event={"ID":"454989b3-ca4f-4b8d-8dc0-17f00a0b55c3","Type":"ContainerStarted","Data":"8ed2cfcf2325c2d8de017179de1c31697112fa998d059407b30e58c7e5822281"} Jul 06 23:43:14.783777 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.783724 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qcg78" event={"ID":"1f61dbcd-7f35-4161-8a71-2a737eed2ded","Type":"ContainerStarted","Data":"778f86ea583b81f183869256d3c8383719d739949287a882eca3228e7bbb177e"} Jul 06 23:43:14.787773 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.787708 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" event={"ID":"753ab644-3d33-4875-bf8d-509d4451c8da","Type":"ContainerStarted","Data":"b00dd9fbec5e05d2f3e12902c4ac10c83704c7ecd8488d2729db2a2dd529fd93"} Jul 06 23:43:14.787949 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.787860 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" event={"ID":"753ab644-3d33-4875-bf8d-509d4451c8da","Type":"ContainerStarted","Data":"31e5f6e2e4e15969fe9d9f7a4c07d4d44a6e4d8c752ba511158e7053ef3c39cd"} Jul 06 23:43:14.787949 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.787911 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" event={"ID":"753ab644-3d33-4875-bf8d-509d4451c8da","Type":"ContainerStarted","Data":"a0b00ea656bf34559364aba6ea153fffe85449e7d69f2d5322d05a0c681892ae"} Jul 06 23:43:14.788039 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.787960 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" event={"ID":"753ab644-3d33-4875-bf8d-509d4451c8da","Type":"ContainerStarted","Data":"7e96d97756fb8df8f31fb0b99e8b68b9c67835be215a2a8d98e10dd69fce0871"} Jul 06 23:43:14.788039 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.788007 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" event={"ID":"753ab644-3d33-4875-bf8d-509d4451c8da","Type":"ContainerStarted","Data":"6d8f7a2afad251572c2788571f970410b5a28fe3d0fe4d0e072155fa21f37784"} Jul 06 23:43:14.788130 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.788054 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" event={"ID":"753ab644-3d33-4875-bf8d-509d4451c8da","Type":"ContainerStarted","Data":"7deba9fe7e321337890a956499570ee836a6ca79f8efa99641d6599c6093cd1b"} Jul 06 23:43:14.789472 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.789441 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" event={"ID":"6dfbb1c6-a6c0-4f03-8789-344efaece614","Type":"ContainerStarted","Data":"ef0b733f114649d89e36cd3b6dc9e56ddc2dae0b2bf3774049556ab2c4c62d06"} Jul 06 23:43:14.791151 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.791122 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6l7np" event={"ID":"a34c708d-ebcf-49e2-ad65-a18ece0d1858","Type":"ContainerStarted","Data":"92d9849a9270113f542a421517c2afee4c3db69d9603db98b940ddba399b6744"} Jul 06 23:43:14.792888 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.792796 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-66.ec2.internal" podStartSLOduration=21.792780331 podStartE2EDuration="21.792780331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:42:56.741321818 +0000 UTC m=+5.633510499" watchObservedRunningTime="2026-07-06 23:43:14.792780331 +0000 UTC m=+23.684969009" Jul 06 23:43:14.812063 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.811992 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zjxbk" podStartSLOduration=3.477020086 podStartE2EDuration="22.81196624s" totalImagesPullingTime="19.334946154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:14.810759087 +0000 UTC m=+23.702947765" watchObservedRunningTime="2026-07-06 23:43:14.81196624 +0000 UTC m=+23.704154919" Jul 06 23:43:14.824556 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.824465 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qcg78" podStartSLOduration=3.491170812 podStartE2EDuration="22.824445838s" totalImagesPullingTime="19.333275026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:14.824048141 +0000 UTC m=+23.716236819" watchObservedRunningTime="2026-07-06 23:43:14.824445838 +0000 UTC m=+23.716634517" Jul 06 23:43:14.838321 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.838265 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kbskp" podStartSLOduration=3.497344629 podStartE2EDuration="22.838245546s" totalImagesPullingTime="19.340900917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:14.837980985 +0000 UTC m=+23.730169726" watchObservedRunningTime="2026-07-06 23:43:14.838245546 +0000 UTC m=+23.730434227" Jul 06 23:43:14.854951 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.854893 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-552zh" podStartSLOduration=3.483971837 podStartE2EDuration="22.854874293s" totalImagesPullingTime="19.370902456s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:14.85444076 +0000 UTC m=+23.746629441" watchObservedRunningTime="2026-07-06 23:43:14.854874293 +0000 UTC m=+23.747062972" Jul 06 23:43:14.867691 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:14.867639 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6l7np" podStartSLOduration=4.532450782 podStartE2EDuration="23.867621401s" totalImagesPullingTime="19.335170619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:14.867586581 +0000 UTC m=+23.759775253" watchObservedRunningTime="2026-07-06 23:43:14.867621401 +0000 UTC m=+23.759810079" Jul 06 23:43:15.001852 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.001802 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:15.102966 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.102928 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:15.114772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.114731 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:43:15.115352 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.115334 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:43:15.404545 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.404515 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:15.501769 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.501729 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:15.557561 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.557515 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:15.557778 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.557519 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:15.557778 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:15.557689 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:15.557778 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:15.557768 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:15.560105 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.560075 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 23:43:15.601987 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.601945 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:15.788335 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.788232 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:15.796719 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.796618 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" event={"ID":"6dfbb1c6-a6c0-4f03-8789-344efaece614","Type":"ContainerStarted","Data":"2c29d3033ca11928f519ce6d3ff873ea707e466fd0a54cdc741256682e9d49a7"} Jul 06 23:43:15.798449 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.798376 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hmzwd" event={"ID":"217e0df0-0e80-4f6e-82df-a4ca808f849c","Type":"ContainerStarted","Data":"c1e23996214a45ffe488ea5cf52125c0a7e09cc5e3bae6782f532bb93b1ca69b"} Jul 06 23:43:15.812990 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:15.812918 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hmzwd" podStartSLOduration=4.484326489 podStartE2EDuration="23.812892734s" totalImagesPullingTime="19.328566245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:15.812714258 +0000 UTC m=+24.704902937" watchObservedRunningTime="2026-07-06 23:43:15.812892734 +0000 UTC m=+24.705081415" Jul 06 23:43:16.111021 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:16.110908 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:16.212234 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:16.212154 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:16.315318 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:16.315274 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:16.542337 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:16.542156 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T23:43:15.560095944Z","UUID":"09f26447-8082-4ff8-ab90-7779eb9430e6","Handler":null,"Name":"","Endpoint":""} Jul 06 23:43:16.543976 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:16.543951 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 23:43:16.544145 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:16.543992 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 23:43:16.557687 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:16.557661 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:16.557908 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:16.557777 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:16.793142 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:16.793060 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:16.804334 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:16.804290 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" event={"ID":"6dfbb1c6-a6c0-4f03-8789-344efaece614","Type":"ContainerStarted","Data":"7f85942aebbe2abd9a13c9292880fafad2230d067f369d67996fdd2de84d21a5"} Jul 06 23:43:16.804877 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:16.804845 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 23:43:16.822508 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:16.822451 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8j5pg" podStartSLOduration=3.433663971 podStartE2EDuration="24.822428966s" totalImagesPullingTime="21.388764995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:42:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:16.821897109 +0000 UTC m=+25.714085813" watchObservedRunningTime="2026-07-06 23:43:16.822428966 +0000 UTC m=+25.714617646" Jul 06 23:43:17.000303 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:17.000249 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:17.558014 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:17.557425 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:17.558014 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:17.557616 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:17.558014 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:17.557668 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:17.558014 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:17.557788 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:17.590653 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:17.590609 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:17.690888 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:17.690848 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:17.810837 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:17.810751 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" event={"ID":"753ab644-3d33-4875-bf8d-509d4451c8da","Type":"ContainerStarted","Data":"8eebb1b755eaf20437b88ab2d27c2fd136b277d3f3c5358a9206c3ef861f19f7"} Jul 06 23:43:17.982935 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:17.982901 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:18.391460 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:18.391421 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:18.557718 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:18.557679 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:18.557911 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:18.557821 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:18.788017 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:18.787894 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:19.186742 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.186499 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:19.388958 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.388925 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:19.557176 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.557125 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:19.557349 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.557125 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:19.557349 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:19.557241 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:19.557463 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:19.557343 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:19.789362 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.789256 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:19.817782 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.817742 2560 generic.go:356] "Generic (PLEG): container finished" podID="2db27634-076c-484d-a1bf-9633194d39c6" containerID="dd1558687306bd0e5bb3e28d86ff796c28dfb377c50ef6cc9928995374194652" exitCode=0 Jul 06 23:43:19.817994 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.817859 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tk9hj" event={"ID":"2db27634-076c-484d-a1bf-9633194d39c6","Type":"ContainerDied","Data":"dd1558687306bd0e5bb3e28d86ff796c28dfb377c50ef6cc9928995374194652"} Jul 06 23:43:19.822002 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.821973 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" event={"ID":"753ab644-3d33-4875-bf8d-509d4451c8da","Type":"ContainerStarted","Data":"daff70c35476a524b9df1106e578475f3a0b0edae8b46c289d600a0677cb2c57"} Jul 06 23:43:19.822356 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.822334 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:43:19.822464 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.822436 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:43:19.840094 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.840052 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:43:19.840272 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.840255 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:43:19.862309 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:19.862259 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" podStartSLOduration=8.431440555 podStartE2EDuration="27.862242247s" totalImagesPullingTime="19.430801692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:19.861170687 +0000 UTC m=+28.753359366" watchObservedRunningTime="2026-07-06 23:43:19.862242247 +0000 UTC m=+28.754430917" Jul 06 23:43:20.390268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:20.390239 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:20.491337 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:20.491024 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:20.557684 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:20.557655 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:20.557864 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:20.557770 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:20.785372 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:20.785298 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:20.831561 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:20.831532 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 23:43:21.193869 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:21.193721 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:21.557596 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:21.557471 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:21.557596 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:21.557591 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:21.557868 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:21.557585 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:21.557868 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:21.557689 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:21.585128 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:21.585094 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:21.787502 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:21.787459 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:21.828354 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:21.828266 2560 generic.go:356] "Generic (PLEG): container finished" podID="2db27634-076c-484d-a1bf-9633194d39c6" containerID="050693afe3dc92b53078560436a8fc8a89972ee9945a0c6688b550601212b453" exitCode=0 Jul 06 23:43:21.828531 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:21.828358 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tk9hj" event={"ID":"2db27634-076c-484d-a1bf-9633194d39c6","Type":"ContainerDied","Data":"050693afe3dc92b53078560436a8fc8a89972ee9945a0c6688b550601212b453"} Jul 06 23:43:21.828531 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:21.828506 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 23:43:21.989002 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:21.988964 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:22.002988 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:22.002955 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:43:22.003268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:22.003101 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 23:43:22.003919 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:22.003898 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6l7np" Jul 06 23:43:22.557432 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:22.557399 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:22.557674 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:22.557532 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:23.419097 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:23.419060 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:43:23.458745 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:23.458711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:23.524369 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:23.524335 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:23.557382 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:23.557347 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:23.557595 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:23.557347 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:23.557595 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:23.557457 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:23.557595 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:23.557535 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:23.835002 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:23.834908 2560 generic.go:356] "Generic (PLEG): container finished" podID="2db27634-076c-484d-a1bf-9633194d39c6" containerID="6fb38916b903657bceecca6185c1117ea27fadfecfd39e650869fcc138584ccd" exitCode=0 Jul 06 23:43:23.835002 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:23.834955 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tk9hj" event={"ID":"2db27634-076c-484d-a1bf-9633194d39c6","Type":"ContainerDied","Data":"6fb38916b903657bceecca6185c1117ea27fadfecfd39e650869fcc138584ccd"} Jul 06 23:43:24.557103 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:24.557061 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:24.557565 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:24.557229 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:25.230968 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:25.230932 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:25.231196 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:25.230998 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:25.231196 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:25.231111 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:43:25.231196 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:25.231127 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:43:25.231196 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:25.231197 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs podName:58cbc7f0-0b10-46b4-8f20-73034d089bc9 nodeName:}" failed. No retries permitted until 2026-07-06 23:43:57.231175644 +0000 UTC m=+66.123364320 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs") pod "network-metrics-daemon-f2vgw" (UID: "58cbc7f0-0b10-46b4-8f20-73034d089bc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:43:25.231412 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:25.231215 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret podName:2a524114-7358-43de-85d9-00e55e281c8a nodeName:}" failed. No retries permitted until 2026-07-06 23:43:57.231208286 +0000 UTC m=+66.123396942 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret") pod "global-pull-secret-syncer-nng68" (UID: "2a524114-7358-43de-85d9-00e55e281c8a") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:43:25.331690 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:25.331646 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdtd\" (UniqueName: \"kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd\") pod \"network-check-target-vlbd8\" (UID: \"d5222093-6d7d-46d1-93f1-47245a02ae3c\") " pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:25.331885 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:25.331817 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:43:25.331885 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:25.331836 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:43:25.331885 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:25.331848 2560 projected.go:196] Error preparing data for projected volume kube-api-access-vmdtd for pod openshift-network-diagnostics/network-check-target-vlbd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:43:25.331998 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:25.331916 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd podName:d5222093-6d7d-46d1-93f1-47245a02ae3c nodeName:}" failed. No retries permitted until 2026-07-06 23:43:57.331896595 +0000 UTC m=+66.224085253 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vmdtd" (UniqueName: "kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd") pod "network-check-target-vlbd8" (UID: "d5222093-6d7d-46d1-93f1-47245a02ae3c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:43:25.557847 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:25.557752 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:25.557847 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:25.557784 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:25.558389 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:25.557879 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:25.558389 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:25.558027 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:26.557167 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:26.557116 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:26.557377 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:26.557247 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:27.333763 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:27.333722 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:27.436390 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:27.436348 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:27.529407 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:27.529354 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:27.561263 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:27.561231 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:27.561443 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:27.561235 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:27.561443 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:27.561379 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:27.561539 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:27.561467 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:27.730408 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:27.730353 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:27.830607 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:27.830549 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:28.464553 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:28.464513 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:28.557447 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:28.557417 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:28.557681 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:28.557528 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:28.559884 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:28.559850 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:28.717484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:28.717398 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:28.852235 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:28.852193 2560 generic.go:356] "Generic (PLEG): container finished" podID="2db27634-076c-484d-a1bf-9633194d39c6" containerID="dc2d1cacf400266225aade184aac3192db68395756a25cc61443b7bff172c3a5" exitCode=0 Jul 06 23:43:28.852400 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:28.852250 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tk9hj" event={"ID":"2db27634-076c-484d-a1bf-9633194d39c6","Type":"ContainerDied","Data":"dc2d1cacf400266225aade184aac3192db68395756a25cc61443b7bff172c3a5"} Jul 06 23:43:28.881073 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:28.881033 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:28.948365 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:28.948335 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:29.130325 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.130117 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f2vgw"] Jul 06 23:43:29.130508 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.130436 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:29.130622 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:29.130597 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:29.132772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.132734 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nng68"] Jul 06 23:43:29.132925 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.132862 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:29.132992 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:29.132965 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:29.133476 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.133454 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vlbd8"] Jul 06 23:43:29.133621 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.133537 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:29.133685 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:29.133642 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:29.297801 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.297713 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:29.428743 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.428709 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:29.535740 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.535693 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:29.603475 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.603355 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:29.672790 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.672747 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:29.857902 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.857815 2560 generic.go:356] "Generic (PLEG): container finished" podID="2db27634-076c-484d-a1bf-9633194d39c6" containerID="03e5d6c53bd395415cdc95a5f6d2494c1beab952ea5ee7f4771323f8dfb49d8f" exitCode=0 Jul 06 23:43:29.857902 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:29.857879 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tk9hj" event={"ID":"2db27634-076c-484d-a1bf-9633194d39c6","Type":"ContainerDied","Data":"03e5d6c53bd395415cdc95a5f6d2494c1beab952ea5ee7f4771323f8dfb49d8f"} Jul 06 23:43:30.100895 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:30.100855 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:30.292148 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:30.292105 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:30.406564 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:30.406524 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:30.557271 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:30.557168 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:30.557271 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:30.557225 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:30.557271 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:30.557178 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:30.558019 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:30.557291 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:30.558019 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:30.557345 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:30.558019 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:30.557428 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:30.863961 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:30.863873 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tk9hj" event={"ID":"2db27634-076c-484d-a1bf-9633194d39c6","Type":"ContainerStarted","Data":"b1e630e8b1c1a814871fe3a84cc58a6196f6a443a6e7da547a981f5fa73e6c4e"} Jul 06 23:43:30.885987 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:30.885935 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tk9hj" podStartSLOduration=9.918165234 podStartE2EDuration="39.885917783s" totalImagesPullingTime="29.967752549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:42:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:30.884969867 +0000 UTC m=+39.777158546" watchObservedRunningTime="2026-07-06 23:43:30.885917783 +0000 UTC m=+39.778106461" Jul 06 23:43:31.007813 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:31.007769 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:31.106937 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:31.106903 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:31.207832 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:31.207799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:31.267286 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:31.267256 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:31.451302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:31.451268 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:32.557054 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:32.557001 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:32.557483 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:32.557001 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:32.557483 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:32.557145 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f2vgw" podUID="58cbc7f0-0b10-46b4-8f20-73034d089bc9" Jul 06 23:43:32.557483 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:32.557001 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:32.557483 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:32.557204 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vlbd8" podUID="d5222093-6d7d-46d1-93f1-47245a02ae3c" Jul 06 23:43:32.557483 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:32.557280 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nng68" podUID="2a524114-7358-43de-85d9-00e55e281c8a" Jul 06 23:43:33.933037 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:33.932954 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-66.ec2.internal" event="NodeReady" Jul 06 23:43:33.933492 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:33.933117 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 23:43:33.981982 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:33.981945 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7d774"] Jul 06 23:43:34.005980 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.005942 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7k55q"] Jul 06 23:43:34.006134 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.006091 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.008405 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.008372 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 23:43:34.008560 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.008480 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 23:43:34.008560 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.008507 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-94g74\"" Jul 06 23:43:34.026097 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.026051 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7d774"] Jul 06 23:43:34.026293 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.026117 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7k55q"] Jul 06 23:43:34.026293 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.026150 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-99g66"] Jul 06 23:43:34.026293 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.026238 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.028555 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.028521 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dsbmx\"" Jul 06 23:43:34.028555 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.028523 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 23:43:34.028765 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.028522 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 23:43:34.028765 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.028585 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 23:43:34.028765 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.028583 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 23:43:34.050176 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.050143 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-99g66"] Jul 06 23:43:34.050334 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.050258 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-99g66" Jul 06 23:43:34.052546 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.052522 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 23:43:34.052836 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.052820 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 23:43:34.052916 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.052869 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9whhz\"" Jul 06 23:43:34.052976 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.052948 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 23:43:34.098941 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.098892 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029cdfae-970b-4894-957c-8e999be93a46-metrics-tls\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.099126 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.098950 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl8jf\" (UniqueName: \"kubernetes.io/projected/029cdfae-970b-4894-957c-8e999be93a46-kube-api-access-zl8jf\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.099126 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.098978 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029cdfae-970b-4894-957c-8e999be93a46-tmp-dir\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.099126 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.099050 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/029cdfae-970b-4894-957c-8e999be93a46-config-volume\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.130376 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.130340 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:34.198003 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.197919 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:34.199887 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.199864 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/029cdfae-970b-4894-957c-8e999be93a46-config-volume\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.199955 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.199900 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4200ae94-dad4-44ee-ba8d-421476fcc12b-crio-socket\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.199955 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.199936 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e133d6cd-0751-4e21-815b-c29b1f6b7327-cert\") pod \"ingress-canary-99g66\" (UID: \"e133d6cd-0751-4e21-815b-c29b1f6b7327\") " pod="openshift-ingress-canary/ingress-canary-99g66" Jul 06 23:43:34.200043 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.199960 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t78nd\" (UniqueName: \"kubernetes.io/projected/4200ae94-dad4-44ee-ba8d-421476fcc12b-kube-api-access-t78nd\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.200043 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.199982 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5lx8\" (UniqueName: \"kubernetes.io/projected/e133d6cd-0751-4e21-815b-c29b1f6b7327-kube-api-access-w5lx8\") pod \"ingress-canary-99g66\" (UID: \"e133d6cd-0751-4e21-815b-c29b1f6b7327\") " pod="openshift-ingress-canary/ingress-canary-99g66" Jul 06 23:43:34.200043 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.200011 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029cdfae-970b-4894-957c-8e999be93a46-metrics-tls\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.200179 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.200138 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4200ae94-dad4-44ee-ba8d-421476fcc12b-data-volume\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.200179 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.200163 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4200ae94-dad4-44ee-ba8d-421476fcc12b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.200274 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.200211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4200ae94-dad4-44ee-ba8d-421476fcc12b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.200274 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.200260 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl8jf\" (UniqueName: \"kubernetes.io/projected/029cdfae-970b-4894-957c-8e999be93a46-kube-api-access-zl8jf\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.200361 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.200280 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029cdfae-970b-4894-957c-8e999be93a46-tmp-dir\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.200480 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.200463 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/029cdfae-970b-4894-957c-8e999be93a46-config-volume\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.200527 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.200499 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/029cdfae-970b-4894-957c-8e999be93a46-tmp-dir\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.204438 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.204409 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/029cdfae-970b-4894-957c-8e999be93a46-metrics-tls\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.209161 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.209138 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl8jf\" (UniqueName: \"kubernetes.io/projected/029cdfae-970b-4894-957c-8e999be93a46-kube-api-access-zl8jf\") pod \"dns-default-7d774\" (UID: \"029cdfae-970b-4894-957c-8e999be93a46\") " pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.301461 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.301414 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5lx8\" (UniqueName: \"kubernetes.io/projected/e133d6cd-0751-4e21-815b-c29b1f6b7327-kube-api-access-w5lx8\") pod \"ingress-canary-99g66\" (UID: \"e133d6cd-0751-4e21-815b-c29b1f6b7327\") " pod="openshift-ingress-canary/ingress-canary-99g66" Jul 06 23:43:34.301731 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.301486 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4200ae94-dad4-44ee-ba8d-421476fcc12b-data-volume\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.301731 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.301509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4200ae94-dad4-44ee-ba8d-421476fcc12b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.301731 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.301531 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4200ae94-dad4-44ee-ba8d-421476fcc12b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.301731 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.301592 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4200ae94-dad4-44ee-ba8d-421476fcc12b-crio-socket\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.301731 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.301644 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e133d6cd-0751-4e21-815b-c29b1f6b7327-cert\") pod \"ingress-canary-99g66\" (UID: \"e133d6cd-0751-4e21-815b-c29b1f6b7327\") " pod="openshift-ingress-canary/ingress-canary-99g66" Jul 06 23:43:34.301731 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.301685 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t78nd\" (UniqueName: \"kubernetes.io/projected/4200ae94-dad4-44ee-ba8d-421476fcc12b-kube-api-access-t78nd\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.301999 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.301914 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4200ae94-dad4-44ee-ba8d-421476fcc12b-crio-socket\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.301999 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.301968 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4200ae94-dad4-44ee-ba8d-421476fcc12b-data-volume\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.302260 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.302231 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4200ae94-dad4-44ee-ba8d-421476fcc12b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.304287 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.304262 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4200ae94-dad4-44ee-ba8d-421476fcc12b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.304449 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.304429 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e133d6cd-0751-4e21-815b-c29b1f6b7327-cert\") pod \"ingress-canary-99g66\" (UID: \"e133d6cd-0751-4e21-815b-c29b1f6b7327\") " pod="openshift-ingress-canary/ingress-canary-99g66" Jul 06 23:43:34.311045 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.311012 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t78nd\" (UniqueName: \"kubernetes.io/projected/4200ae94-dad4-44ee-ba8d-421476fcc12b-kube-api-access-t78nd\") pod \"insights-runtime-extractor-7k55q\" (UID: \"4200ae94-dad4-44ee-ba8d-421476fcc12b\") " pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.311157 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.311128 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5lx8\" (UniqueName: \"kubernetes.io/projected/e133d6cd-0751-4e21-815b-c29b1f6b7327-kube-api-access-w5lx8\") pod \"ingress-canary-99g66\" (UID: \"e133d6cd-0751-4e21-815b-c29b1f6b7327\") " pod="openshift-ingress-canary/ingress-canary-99g66" Jul 06 23:43:34.315015 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.314974 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7d774" Jul 06 23:43:34.335283 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.335252 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7k55q" Jul 06 23:43:34.359208 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.359173 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-99g66" Jul 06 23:43:34.541926 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.541895 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7d774"] Jul 06 23:43:34.545934 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:34.545888 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029cdfae_970b_4894_957c_8e999be93a46.slice/crio-722c3f59007334606500c8c4e066738962a46a439883a0db485a26830cc010b8 WatchSource:0}: Error finding container 722c3f59007334606500c8c4e066738962a46a439883a0db485a26830cc010b8: Status 404 returned error can't find the container with id 722c3f59007334606500c8c4e066738962a46a439883a0db485a26830cc010b8 Jul 06 23:43:34.554352 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.554261 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7k55q"] Jul 06 23:43:34.557241 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.557215 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:34.557356 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.557338 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:34.557400 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.557389 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:34.557442 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:34.557415 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4200ae94_dad4_44ee_ba8d_421476fcc12b.slice/crio-6a4bf125e00ffaef60695e428d1fa825d0375d568ee3b6ef9f57ce515a2e64fe WatchSource:0}: Error finding container 6a4bf125e00ffaef60695e428d1fa825d0375d568ee3b6ef9f57ce515a2e64fe: Status 404 returned error can't find the container with id 6a4bf125e00ffaef60695e428d1fa825d0375d568ee3b6ef9f57ce515a2e64fe Jul 06 23:43:34.559708 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.559685 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:43:34.560379 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.560357 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pwj6r\"" Jul 06 23:43:34.560481 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.560396 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:43:34.560481 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.560423 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:43:34.560481 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.560427 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:43:34.560744 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.560495 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wsqvs\"" Jul 06 23:43:34.570348 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.570319 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-99g66"] Jul 06 23:43:34.574087 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:34.574052 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode133d6cd_0751_4e21_815b_c29b1f6b7327.slice/crio-4b7b3be3a92264fe684fee925754988e27eb79aa1eb26e20edbc1382c9f3d5d6 WatchSource:0}: Error finding container 4b7b3be3a92264fe684fee925754988e27eb79aa1eb26e20edbc1382c9f3d5d6: Status 404 returned error can't find the container with id 4b7b3be3a92264fe684fee925754988e27eb79aa1eb26e20edbc1382c9f3d5d6 Jul 06 23:43:34.875361 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.875260 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7k55q" event={"ID":"4200ae94-dad4-44ee-ba8d-421476fcc12b","Type":"ContainerStarted","Data":"6a4bf125e00ffaef60695e428d1fa825d0375d568ee3b6ef9f57ce515a2e64fe"} Jul 06 23:43:34.876507 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.876475 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7d774" event={"ID":"029cdfae-970b-4894-957c-8e999be93a46","Type":"ContainerStarted","Data":"722c3f59007334606500c8c4e066738962a46a439883a0db485a26830cc010b8"} Jul 06 23:43:34.877630 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:34.877603 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-99g66" event={"ID":"e133d6cd-0751-4e21-815b-c29b1f6b7327","Type":"ContainerStarted","Data":"4b7b3be3a92264fe684fee925754988e27eb79aa1eb26e20edbc1382c9f3d5d6"} Jul 06 23:43:37.515631 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.515550 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:37.517944 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.517912 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8bf8c8f46-tvptr"] Jul 06 23:43:37.566805 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.566767 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:37.596017 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.595976 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.599628 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.599538 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9bqw6\"" Jul 06 23:43:37.599804 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.599728 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 23:43:37.599804 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.599798 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 23:43:37.600472 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.599957 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 23:43:37.600472 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.600022 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 23:43:37.600472 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.600036 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 23:43:37.600472 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.600279 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 23:43:37.600472 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.600321 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 23:43:37.601027 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.601005 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8bf8c8f46-tvptr"] Jul 06 23:43:37.610345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.610313 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 23:43:37.733325 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.733287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-oauth-serving-cert\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.733325 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.733334 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-serving-cert\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.733597 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.733352 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-oauth-config\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.733597 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.733367 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-config\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.733597 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.733385 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wg88\" (UniqueName: \"kubernetes.io/projected/0e736abd-483f-4266-a13f-ba1d815ea7d6-kube-api-access-6wg88\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.733597 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.733404 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-trusted-ca-bundle\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.733597 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.733554 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-service-ca\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.834250 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.834138 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-serving-cert\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.834250 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.834182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-oauth-config\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.834250 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.834199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-config\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.834250 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.834216 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wg88\" (UniqueName: \"kubernetes.io/projected/0e736abd-483f-4266-a13f-ba1d815ea7d6-kube-api-access-6wg88\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.834607 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.834304 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-trusted-ca-bundle\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.834607 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.834409 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-service-ca\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.834607 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.834479 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-oauth-serving-cert\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.835117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.835087 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-config\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.835268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.835164 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-oauth-serving-cert\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.835268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.835214 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-trusted-ca-bundle\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.835623 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.835599 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-service-ca\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.836921 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.836898 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-oauth-config\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.837101 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.837081 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-serving-cert\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.841792 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.841767 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wg88\" (UniqueName: \"kubernetes.io/projected/0e736abd-483f-4266-a13f-ba1d815ea7d6-kube-api-access-6wg88\") pod \"console-8bf8c8f46-tvptr\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:37.909813 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:37.909771 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:38.457153 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:38.457112 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:38.513840 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:38.513806 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:40.161429 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.161386 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:40.263531 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.263480 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:40.316436 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.316405 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx"] Jul 06 23:43:40.336431 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.336402 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx"] Jul 06 23:43:40.336648 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.336523 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx" Jul 06 23:43:40.337737 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.337713 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:40.338975 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.338953 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 23:43:40.338975 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.338972 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-ljx2q\"" Jul 06 23:43:40.354637 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.354606 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/deef3342-f965-4f87-8a30-8ae7007a8a31-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-288bx\" (UID: \"deef3342-f965-4f87-8a30-8ae7007a8a31\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx" Jul 06 23:43:40.455794 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.455758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/deef3342-f965-4f87-8a30-8ae7007a8a31-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-288bx\" (UID: \"deef3342-f965-4f87-8a30-8ae7007a8a31\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx" Jul 06 23:43:40.456004 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:40.455919 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 06 23:43:40.456004 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:40.455988 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deef3342-f965-4f87-8a30-8ae7007a8a31-tls-certificates podName:deef3342-f965-4f87-8a30-8ae7007a8a31 nodeName:}" failed. No retries permitted until 2026-07-06 23:43:40.955970916 +0000 UTC m=+49.848159574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/deef3342-f965-4f87-8a30-8ae7007a8a31-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-288bx" (UID: "deef3342-f965-4f87-8a30-8ae7007a8a31") : secret "prometheus-operator-admission-webhook-tls" not found Jul 06 23:43:40.827331 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.827253 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:40.959501 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.959476 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/deef3342-f965-4f87-8a30-8ae7007a8a31-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-288bx\" (UID: \"deef3342-f965-4f87-8a30-8ae7007a8a31\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx" Jul 06 23:43:40.962720 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.962690 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/deef3342-f965-4f87-8a30-8ae7007a8a31-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-288bx\" (UID: \"deef3342-f965-4f87-8a30-8ae7007a8a31\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx" Jul 06 23:43:40.980206 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:40.980179 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8bf8c8f46-tvptr"] Jul 06 23:43:41.040954 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:41.040920 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e736abd_483f_4266_a13f_ba1d815ea7d6.slice/crio-35880a46be07e419f9c7b25d212754b21596b1a1972530f7ed183c153013dbbb WatchSource:0}: Error finding container 35880a46be07e419f9c7b25d212754b21596b1a1972530f7ed183c153013dbbb: Status 404 returned error can't find the container with id 35880a46be07e419f9c7b25d212754b21596b1a1972530f7ed183c153013dbbb Jul 06 23:43:41.246412 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:41.246368 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx" Jul 06 23:43:41.386643 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:41.386613 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx"] Jul 06 23:43:41.390191 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:41.390147 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeef3342_f965_4f87_8a30_8ae7007a8a31.slice/crio-e582e7a6ae04ded99ec0bec4ac3432b8ab2a3f5d09ded82e14e426ab47279daa WatchSource:0}: Error finding container e582e7a6ae04ded99ec0bec4ac3432b8ab2a3f5d09ded82e14e426ab47279daa: Status 404 returned error can't find the container with id e582e7a6ae04ded99ec0bec4ac3432b8ab2a3f5d09ded82e14e426ab47279daa Jul 06 23:43:41.902545 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:41.902500 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7d774" event={"ID":"029cdfae-970b-4894-957c-8e999be93a46","Type":"ContainerStarted","Data":"df59c247e8f4873d63a81ae21a9d7af51df46ed700b6e41134cbbb0d5154de54"} Jul 06 23:43:41.902765 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:41.902592 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7d774" event={"ID":"029cdfae-970b-4894-957c-8e999be93a46","Type":"ContainerStarted","Data":"466f53140585199bcc8455d8f0ecad9f400377beb540602c4509db37d20de353"} Jul 06 23:43:41.902860 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:41.902838 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7d774" Jul 06 23:43:41.904876 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:41.904842 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-99g66" event={"ID":"e133d6cd-0751-4e21-815b-c29b1f6b7327","Type":"ContainerStarted","Data":"554a061452423f81cd9b338add59799135cf4c58e44af3c93c4b18a58765a65c"} Jul 06 23:43:41.906457 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:41.906420 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7k55q" event={"ID":"4200ae94-dad4-44ee-ba8d-421476fcc12b","Type":"ContainerStarted","Data":"c96871f584d80874b90bd5ec0015b6b944ba6b2edbbda3ed121462d80df5916e"} Jul 06 23:43:41.907905 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:41.907875 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx" event={"ID":"deef3342-f965-4f87-8a30-8ae7007a8a31","Type":"ContainerStarted","Data":"e582e7a6ae04ded99ec0bec4ac3432b8ab2a3f5d09ded82e14e426ab47279daa"} Jul 06 23:43:41.909365 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:41.909334 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bf8c8f46-tvptr" event={"ID":"0e736abd-483f-4266-a13f-ba1d815ea7d6","Type":"ContainerStarted","Data":"35880a46be07e419f9c7b25d212754b21596b1a1972530f7ed183c153013dbbb"} Jul 06 23:43:41.920607 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:41.919995 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7d774" podStartSLOduration=2.624325288 podStartE2EDuration="8.91997715s" totalImagesPullingTime="6.295651862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:41.918440979 +0000 UTC m=+50.810629659" watchObservedRunningTime="2026-07-06 23:43:41.91997715 +0000 UTC m=+50.812165818" Jul 06 23:43:41.932829 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:41.932761 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-99g66" podStartSLOduration=2.6699312539999998 podStartE2EDuration="8.932742622s" totalImagesPullingTime="6.262811368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:43:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:41.932120732 +0000 UTC m=+50.824309413" watchObservedRunningTime="2026-07-06 23:43:41.932742622 +0000 UTC m=+50.824931299" Jul 06 23:43:42.198042 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:42.197941 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:42.289855 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:42.289820 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:42.595134 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:42.595094 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:42.694997 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:42.694959 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:42.748086 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:42.748033 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:42.914163 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:42.914120 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7k55q" event={"ID":"4200ae94-dad4-44ee-ba8d-421476fcc12b","Type":"ContainerStarted","Data":"4c34fa65690a94a2ecb37b5cf82f664ce4561990e2cdbf7d752d30a8d9f6c924"} Jul 06 23:43:44.927215 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:44.927116 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx" event={"ID":"deef3342-f965-4f87-8a30-8ae7007a8a31","Type":"ContainerStarted","Data":"f800845a4eba1a5d6e398457c49b92d6ab972a726055880c09eb5157d9f3afa7"} Jul 06 23:43:44.927731 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:44.927431 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx" Jul 06 23:43:44.929108 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:44.929077 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bf8c8f46-tvptr" event={"ID":"0e736abd-483f-4266-a13f-ba1d815ea7d6","Type":"ContainerStarted","Data":"6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598"} Jul 06 23:43:44.934074 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:44.934034 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx" Jul 06 23:43:44.942055 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:44.941988 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-288bx" podStartSLOduration=2.037683983 podStartE2EDuration="4.941967443s" totalImagesPullingTime="2.90428346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:43:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:44.941085281 +0000 UTC m=+53.833273976" watchObservedRunningTime="2026-07-06 23:43:44.941967443 +0000 UTC m=+53.834156121" Jul 06 23:43:44.975047 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:44.974975 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8bf8c8f46-tvptr" podStartSLOduration=4.400571484 podStartE2EDuration="7.974952968s" totalImagesPullingTime="3.574381484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:43:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:44.972663425 +0000 UTC m=+53.864852107" watchObservedRunningTime="2026-07-06 23:43:44.974952968 +0000 UTC m=+53.867141646" Jul 06 23:43:45.339533 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:45.339487 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:45.440225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:45.440181 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:45.541810 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:45.541762 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:46.328693 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:46.328610 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:46.399374 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:46.399336 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:46.948432 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:46.948396 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:47.048579 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:47.048531 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:47.111099 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:47.111055 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:47.910249 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:47.910211 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:47.910694 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:47.910283 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:47.911691 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:47.911661 2560 patch_prober.go:28] interesting pod/console-8bf8c8f46-tvptr container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 06 23:43:47.911828 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:47.911712 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-8bf8c8f46-tvptr" podUID="0e736abd-483f-4266-a13f-ba1d815ea7d6" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 06 23:43:47.940995 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:47.940960 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7k55q" event={"ID":"4200ae94-dad4-44ee-ba8d-421476fcc12b","Type":"ContainerStarted","Data":"fc4505a8aa7f1340bc1872a161e776993b253ca8c654af55445880e1f61388d1"} Jul 06 23:43:47.962588 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:47.962520 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7k55q" podStartSLOduration=2.615013975 podStartE2EDuration="14.962500251s" totalImagesPullingTime="12.347486276s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:43:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:47.961528988 +0000 UTC m=+56.853717672" watchObservedRunningTime="2026-07-06 23:43:47.962500251 +0000 UTC m=+56.854688928" Jul 06 23:43:50.227975 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:50.227935 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:50.375463 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:50.375429 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:50.427282 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:50.427244 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:51.503980 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:51.503942 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:51.916351 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:51.916273 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7d774" Jul 06 23:43:52.383810 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:52.383775 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:52.445552 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:52.445518 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:53.431430 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.431396 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92dbw" Jul 06 23:43:53.497898 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.497851 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:53.593604 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.593551 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:53.723566 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.723527 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:53.803034 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.802997 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk"] Jul 06 23:43:53.805984 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.805966 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:53.808282 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.808250 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 23:43:53.808747 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.808732 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 23:43:53.808868 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.808844 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 23:43:53.808932 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.808871 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-nq9s4\"" Jul 06 23:43:53.809308 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.809291 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 23:43:53.809391 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.809315 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 23:43:53.815283 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.815257 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk"] Jul 06 23:43:53.827681 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.827649 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:53.856637 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.856600 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nbl2d"] Jul 06 23:43:53.858796 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.858771 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:53.860309 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.860268 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:53.860477 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.860335 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7930878-bc87-4bf7-a713-9863d32929ca-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:53.860477 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.860376 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpp88\" (UniqueName: \"kubernetes.io/projected/d7930878-bc87-4bf7-a713-9863d32929ca-kube-api-access-tpp88\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:53.860477 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.860402 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:53.861083 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.861055 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 23:43:53.861231 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.861216 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 23:43:53.861362 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.861347 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 23:43:53.861616 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.861601 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jvgrg\"" Jul 06 23:43:53.961052 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961012 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f761b63d-bb73-4e87-890f-cef98fccf41c-root\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:53.961052 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961055 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:53.961268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961085 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-tls\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:53.961268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961131 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpp88\" (UniqueName: \"kubernetes.io/projected/d7930878-bc87-4bf7-a713-9863d32929ca-kube-api-access-tpp88\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:53.961268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961151 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnrhl\" (UniqueName: \"kubernetes.io/projected/f761b63d-bb73-4e87-890f-cef98fccf41c-kube-api-access-wnrhl\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:53.961268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961200 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-textfile\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:53.961268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961217 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:53.961268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961239 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f761b63d-bb73-4e87-890f-cef98fccf41c-metrics-client-ca\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:53.961533 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961302 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7930878-bc87-4bf7-a713-9863d32929ca-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:53.961533 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961473 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:53.961533 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f761b63d-bb73-4e87-890f-cef98fccf41c-sys\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:53.961704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-wtmp\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:53.961704 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.961590 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-accelerators-collector-config\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:53.961704 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:53.961602 2560 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 23:43:53.961704 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:53.961668 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-tls podName:d7930878-bc87-4bf7-a713-9863d32929ca nodeName:}" failed. No retries permitted until 2026-07-06 23:43:54.461649415 +0000 UTC m=+63.353838074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-fp8mk" (UID: "d7930878-bc87-4bf7-a713-9863d32929ca") : secret "openshift-state-metrics-tls" not found Jul 06 23:43:53.962143 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.962120 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7930878-bc87-4bf7-a713-9863d32929ca-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:53.963915 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.963889 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:53.969293 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:53.969270 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpp88\" (UniqueName: \"kubernetes.io/projected/d7930878-bc87-4bf7-a713-9863d32929ca-kube-api-access-tpp88\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:54.015448 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.015328 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:54.062654 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.062616 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f761b63d-bb73-4e87-890f-cef98fccf41c-root\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.062790 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.062658 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.062790 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.062677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-tls\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.062790 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.062701 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnrhl\" (UniqueName: \"kubernetes.io/projected/f761b63d-bb73-4e87-890f-cef98fccf41c-kube-api-access-wnrhl\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.062790 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.062739 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f761b63d-bb73-4e87-890f-cef98fccf41c-root\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.062790 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.062768 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-textfile\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.063026 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.062801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f761b63d-bb73-4e87-890f-cef98fccf41c-metrics-client-ca\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.063026 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:54.062805 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 23:43:54.063026 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:54.062869 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-tls podName:f761b63d-bb73-4e87-890f-cef98fccf41c nodeName:}" failed. No retries permitted until 2026-07-06 23:43:54.562850887 +0000 UTC m=+63.455039558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-tls") pod "node-exporter-nbl2d" (UID: "f761b63d-bb73-4e87-890f-cef98fccf41c") : secret "node-exporter-tls" not found Jul 06 23:43:54.063026 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.062885 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f761b63d-bb73-4e87-890f-cef98fccf41c-sys\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.063026 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.062929 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-wtmp\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.063026 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.062955 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-accelerators-collector-config\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.063026 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.062975 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f761b63d-bb73-4e87-890f-cef98fccf41c-sys\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.063320 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.063093 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-wtmp\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.063320 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.063187 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-textfile\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.063476 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.063456 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f761b63d-bb73-4e87-890f-cef98fccf41c-metrics-client-ca\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.063523 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.063509 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-accelerators-collector-config\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.065219 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.065201 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.073136 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.073109 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnrhl\" (UniqueName: \"kubernetes.io/projected/f761b63d-bb73-4e87-890f-cef98fccf41c-kube-api-access-wnrhl\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.110853 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.110821 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:54.212958 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.212917 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:54.315023 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.314921 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:54.411756 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.411707 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:54.465159 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.465123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:54.465553 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:54.465287 2560 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 23:43:54.465553 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:54.465355 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-tls podName:d7930878-bc87-4bf7-a713-9863d32929ca nodeName:}" failed. No retries permitted until 2026-07-06 23:43:55.4653365 +0000 UTC m=+64.357525159 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-fp8mk" (UID: "d7930878-bc87-4bf7-a713-9863d32929ca") : secret "openshift-state-metrics-tls" not found Jul 06 23:43:54.566246 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.566146 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-tls\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.568682 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.568663 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f761b63d-bb73-4e87-890f-cef98fccf41c-node-exporter-tls\") pod \"node-exporter-nbl2d\" (UID: \"f761b63d-bb73-4e87-890f-cef98fccf41c\") " pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.639242 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.639202 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:54.739688 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.739653 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:54.768205 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.768170 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nbl2d" Jul 06 23:43:54.777719 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:54.777689 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf761b63d_bb73_4e87_890f_cef98fccf41c.slice/crio-01832b003976af35397d0a8202b37df308dc372dacc8d0c033a99f873750b173 WatchSource:0}: Error finding container 01832b003976af35397d0a8202b37df308dc372dacc8d0c033a99f873750b173: Status 404 returned error can't find the container with id 01832b003976af35397d0a8202b37df308dc372dacc8d0c033a99f873750b173 Jul 06 23:43:54.814018 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.813985 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:54.814507 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.814485 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 23:43:54.817550 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.817504 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.819902 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.819878 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 23:43:54.820050 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.819916 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 23:43:54.820050 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.819987 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 23:43:54.820050 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.820016 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 23:43:54.820232 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.820138 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 23:43:54.820323 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.820308 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 23:43:54.820559 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.820518 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 23:43:54.820764 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.820746 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 23:43:54.820834 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.820754 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 23:43:54.820834 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.820799 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mtxgn\"" Jul 06 23:43:54.834009 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.833986 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 23:43:54.869439 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869405 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-web-config\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.869649 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869458 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.869649 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869477 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc2v9\" (UniqueName: \"kubernetes.io/projected/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-kube-api-access-qc2v9\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.869649 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869531 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.869649 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869564 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.869900 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869673 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.869900 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869713 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-config-volume\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.869900 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869762 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.869900 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869797 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-config-out\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.869900 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.869900 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869857 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.869900 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869890 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.870131 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.869917 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.954816 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.954780 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:54.963145 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.963116 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nbl2d" event={"ID":"f761b63d-bb73-4e87-890f-cef98fccf41c","Type":"ContainerStarted","Data":"01832b003976af35397d0a8202b37df308dc372dacc8d0c033a99f873750b173"} Jul 06 23:43:54.970423 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.970397 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-web-config\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.970513 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.970464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.970513 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.970490 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc2v9\" (UniqueName: \"kubernetes.io/projected/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-kube-api-access-qc2v9\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.970640 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.970519 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.970640 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.970543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.970731 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:54.970668 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-alertmanager-trusted-ca-bundle podName:d999c3e1-d0a7-4d69-96eb-f954ba0dafd5 nodeName:}" failed. No retries permitted until 2026-07-06 23:43:55.470643977 +0000 UTC m=+64.362832650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "d999c3e1-d0a7-4d69-96eb-f954ba0dafd5") : configmap references non-existent config key: ca-bundle.crt Jul 06 23:43:54.970793 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.970726 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.970793 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.970775 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-config-volume\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.970890 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.970813 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.970982 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.970958 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-config-out\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.971067 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.971000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.971121 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.971063 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.971174 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.971101 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.971174 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.971148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.971654 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:54.971492 2560 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 06 23:43:54.971654 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.971544 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.971654 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:54.971549 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-main-tls podName:d999c3e1-d0a7-4d69-96eb-f954ba0dafd5 nodeName:}" failed. No retries permitted until 2026-07-06 23:43:55.471532462 +0000 UTC m=+64.363721135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d999c3e1-d0a7-4d69-96eb-f954ba0dafd5") : secret "alertmanager-main-tls" not found Jul 06 23:43:54.972129 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.971671 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.974456 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.974183 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.974456 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.974183 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.974456 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.974311 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-web-config\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.974456 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.974339 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.974456 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.974396 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-config-volume\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.974456 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.974431 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-config-out\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.974456 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.974441 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.975489 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.975467 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:54.979444 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:54.979418 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc2v9\" (UniqueName: \"kubernetes.io/projected/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-kube-api-access-qc2v9\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:55.057270 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.057236 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:55.255219 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.255182 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:55.328638 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.328608 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:55.475994 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.475958 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:55.476360 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.476014 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:55.476360 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.476202 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:55.476789 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.476760 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:55.478504 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.478486 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d999c3e1-d0a7-4d69-96eb-f954ba0dafd5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:55.478589 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.478552 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7930878-bc87-4bf7-a713-9863d32929ca-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fp8mk\" (UID: \"d7930878-bc87-4bf7-a713-9863d32929ca\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:55.615542 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.615467 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" Jul 06 23:43:55.727239 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.727206 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:43:55.737119 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.737088 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7b77bb6784-dwf8f"] Jul 06 23:43:55.739923 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.739901 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.742553 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.742534 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 23:43:55.742692 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.742595 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 23:43:55.742692 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.742627 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 23:43:55.742692 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.742635 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8hin5h5s2plnm\"" Jul 06 23:43:55.742692 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.742687 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 23:43:55.742863 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.742711 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 23:43:55.742983 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.742956 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-s97lb\"" Jul 06 23:43:55.752228 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.752202 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b77bb6784-dwf8f"] Jul 06 23:43:55.779749 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.779341 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.779749 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.779394 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cmgf\" (UniqueName: \"kubernetes.io/projected/dec7b01b-e69f-423b-a48e-6919828a859f-kube-api-access-2cmgf\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.779749 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.779429 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dec7b01b-e69f-423b-a48e-6919828a859f-metrics-client-ca\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.779749 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.779471 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.779749 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.779497 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.779749 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.779524 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-grpc-tls\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.779749 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.779556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.779749 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.779649 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-tls\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.880848 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.880796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-tls\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.880977 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.880872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.880977 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.880911 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cmgf\" (UniqueName: \"kubernetes.io/projected/dec7b01b-e69f-423b-a48e-6919828a859f-kube-api-access-2cmgf\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.880977 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.880944 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dec7b01b-e69f-423b-a48e-6919828a859f-metrics-client-ca\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.881124 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.880982 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.881124 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.881008 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.881124 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.881038 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-grpc-tls\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.881124 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.881079 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.882464 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.881801 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dec7b01b-e69f-423b-a48e-6919828a859f-metrics-client-ca\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.884628 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.884598 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-tls\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.885804 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.885770 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.886005 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.885933 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.886005 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.885975 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-grpc-tls\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.886472 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.886449 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.886885 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.886863 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dec7b01b-e69f-423b-a48e-6919828a859f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.889758 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.889742 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cmgf\" (UniqueName: \"kubernetes.io/projected/dec7b01b-e69f-423b-a48e-6919828a859f-kube-api-access-2cmgf\") pod \"thanos-querier-7b77bb6784-dwf8f\" (UID: \"dec7b01b-e69f-423b-a48e-6919828a859f\") " pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:55.934804 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.934773 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 23:43:55.939755 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:55.939724 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd999c3e1_d0a7_4d69_96eb_f954ba0dafd5.slice/crio-31a356519fab4848e2ff9c2b4cff605199cfd6c8d35e9a03fa1503f901ed37dc WatchSource:0}: Error finding container 31a356519fab4848e2ff9c2b4cff605199cfd6c8d35e9a03fa1503f901ed37dc: Status 404 returned error can't find the container with id 31a356519fab4848e2ff9c2b4cff605199cfd6c8d35e9a03fa1503f901ed37dc Jul 06 23:43:55.949734 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.949711 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk"] Jul 06 23:43:55.952845 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:55.952805 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7930878_bc87_4bf7_a713_9863d32929ca.slice/crio-47566d920b0818ddfc866b424832f1ffe4efc9dd49291a7a821c353400aa4f0c WatchSource:0}: Error finding container 47566d920b0818ddfc866b424832f1ffe4efc9dd49291a7a821c353400aa4f0c: Status 404 returned error can't find the container with id 47566d920b0818ddfc866b424832f1ffe4efc9dd49291a7a821c353400aa4f0c Jul 06 23:43:55.973103 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.973060 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" event={"ID":"d7930878-bc87-4bf7-a713-9863d32929ca","Type":"ContainerStarted","Data":"47566d920b0818ddfc866b424832f1ffe4efc9dd49291a7a821c353400aa4f0c"} Jul 06 23:43:55.974546 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.974518 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5","Type":"ContainerStarted","Data":"31a356519fab4848e2ff9c2b4cff605199cfd6c8d35e9a03fa1503f901ed37dc"} Jul 06 23:43:55.976198 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:55.976175 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nbl2d" event={"ID":"f761b63d-bb73-4e87-890f-cef98fccf41c","Type":"ContainerStarted","Data":"021f066395c795dc72bc32025982bf604bc223f0af16cdb2fb88b0b5fa021ed4"} Jul 06 23:43:56.052049 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.052014 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:43:56.189450 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.189417 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:56.190139 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.190117 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b77bb6784-dwf8f"] Jul 06 23:43:56.193587 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:56.193552 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddec7b01b_e69f_423b_a48e_6919828a859f.slice/crio-63d597667e98ff27b3e22aa9eb5084b358d3ff5a000ae5d144334dd2880a5f6f WatchSource:0}: Error finding container 63d597667e98ff27b3e22aa9eb5084b358d3ff5a000ae5d144334dd2880a5f6f: Status 404 returned error can't find the container with id 63d597667e98ff27b3e22aa9eb5084b358d3ff5a000ae5d144334dd2880a5f6f Jul 06 23:43:56.268789 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.268757 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:56.661516 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.661481 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:56.715432 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.715396 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:56.859677 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.859641 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:56.984203 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.984159 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" event={"ID":"d7930878-bc87-4bf7-a713-9863d32929ca","Type":"ContainerStarted","Data":"5176cb8a6b446a9ea8b9adb7296920cfbe4b9840dc28fcf9d70e5ac191b031ba"} Jul 06 23:43:56.984381 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.984244 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" event={"ID":"d7930878-bc87-4bf7-a713-9863d32929ca","Type":"ContainerStarted","Data":"479e7607e849c5f7cbe15f0c90af4b64d18ca3e716dbb56de1db0e1a16a6145b"} Jul 06 23:43:56.985775 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.985733 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" event={"ID":"dec7b01b-e69f-423b-a48e-6919828a859f","Type":"ContainerStarted","Data":"63d597667e98ff27b3e22aa9eb5084b358d3ff5a000ae5d144334dd2880a5f6f"} Jul 06 23:43:56.987618 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.987595 2560 generic.go:356] "Generic (PLEG): container finished" podID="f761b63d-bb73-4e87-890f-cef98fccf41c" containerID="021f066395c795dc72bc32025982bf604bc223f0af16cdb2fb88b0b5fa021ed4" exitCode=0 Jul 06 23:43:56.987825 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:56.987641 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nbl2d" event={"ID":"f761b63d-bb73-4e87-890f-cef98fccf41c","Type":"ContainerDied","Data":"021f066395c795dc72bc32025982bf604bc223f0af16cdb2fb88b0b5fa021ed4"} Jul 06 23:43:57.140759 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.140719 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:57.293329 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.293248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:57.293329 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.293305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:57.295846 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.295822 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:43:57.295990 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.295831 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:43:57.306723 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.306695 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58cbc7f0-0b10-46b4-8f20-73034d089bc9-metrics-certs\") pod \"network-metrics-daemon-f2vgw\" (UID: \"58cbc7f0-0b10-46b4-8f20-73034d089bc9\") " pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:57.306856 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.306694 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a524114-7358-43de-85d9-00e55e281c8a-original-pull-secret\") pod \"global-pull-secret-syncer-nng68\" (UID: \"2a524114-7358-43de-85d9-00e55e281c8a\") " pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:57.372916 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.372872 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nng68" Jul 06 23:43:57.381815 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.381597 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wsqvs\"" Jul 06 23:43:57.389793 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.389762 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f2vgw" Jul 06 23:43:57.394492 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.394463 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdtd\" (UniqueName: \"kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd\") pod \"network-check-target-vlbd8\" (UID: \"d5222093-6d7d-46d1-93f1-47245a02ae3c\") " pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:57.397256 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.397050 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:43:57.406942 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.406917 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:43:57.419865 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.419834 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmdtd\" (UniqueName: \"kubernetes.io/projected/d5222093-6d7d-46d1-93f1-47245a02ae3c-kube-api-access-vmdtd\") pod \"network-check-target-vlbd8\" (UID: \"d5222093-6d7d-46d1-93f1-47245a02ae3c\") " pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:57.661277 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.661069 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nng68"] Jul 06 23:43:57.674078 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.674039 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f2vgw"] Jul 06 23:43:57.677758 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:57.677728 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58cbc7f0_0b10_46b4_8f20_73034d089bc9.slice/crio-4ddcb3836b3b22fd7258fb96a2fd67f9361308bc095fff3907c702845ecd8ec1 WatchSource:0}: Error finding container 4ddcb3836b3b22fd7258fb96a2fd67f9361308bc095fff3907c702845ecd8ec1: Status 404 returned error can't find the container with id 4ddcb3836b3b22fd7258fb96a2fd67f9361308bc095fff3907c702845ecd8ec1 Jul 06 23:43:57.678208 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:57.678182 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a524114_7358_43de_85d9_00e55e281c8a.slice/crio-b4667d98d0d14bf09ff86e6b33a4951f62be461adfdf26c104324bb774e0d2c3 WatchSource:0}: Error finding container b4667d98d0d14bf09ff86e6b33a4951f62be461adfdf26c104324bb774e0d2c3: Status 404 returned error can't find the container with id b4667d98d0d14bf09ff86e6b33a4951f62be461adfdf26c104324bb774e0d2c3 Jul 06 23:43:57.687075 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.687052 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pwj6r\"" Jul 06 23:43:57.695620 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.695600 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:43:57.837813 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.837733 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vlbd8"] Jul 06 23:43:57.914589 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.914536 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:57.918849 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.918823 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:43:57.993186 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.993154 2560 generic.go:356] "Generic (PLEG): container finished" podID="d999c3e1-d0a7-4d69-96eb-f954ba0dafd5" containerID="5dff69415106ee09cbb8688c92701d27d68538dc5d1e577baeaa5577f308ec9c" exitCode=0 Jul 06 23:43:57.993351 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.993231 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5","Type":"ContainerDied","Data":"5dff69415106ee09cbb8688c92701d27d68538dc5d1e577baeaa5577f308ec9c"} Jul 06 23:43:57.994655 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.994626 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nng68" event={"ID":"2a524114-7358-43de-85d9-00e55e281c8a","Type":"ContainerStarted","Data":"b4667d98d0d14bf09ff86e6b33a4951f62be461adfdf26c104324bb774e0d2c3"} Jul 06 23:43:57.997232 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.997212 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nbl2d" event={"ID":"f761b63d-bb73-4e87-890f-cef98fccf41c","Type":"ContainerStarted","Data":"0a9e5da9b6ef2413b17798b62eaed9ec668a814d872f6bde13ab57e489314d0e"} Jul 06 23:43:57.997310 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.997247 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nbl2d" event={"ID":"f761b63d-bb73-4e87-890f-cef98fccf41c","Type":"ContainerStarted","Data":"a00fb17f265f8f5f335c5877644a52c9ff37eec6d00f524463faaecc99d1ee0f"} Jul 06 23:43:57.999096 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:57.999072 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f2vgw" event={"ID":"58cbc7f0-0b10-46b4-8f20-73034d089bc9","Type":"ContainerStarted","Data":"4ddcb3836b3b22fd7258fb96a2fd67f9361308bc095fff3907c702845ecd8ec1"} Jul 06 23:43:58.007487 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.007110 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" event={"ID":"d7930878-bc87-4bf7-a713-9863d32929ca","Type":"ContainerStarted","Data":"2f3a00c8e69f869151c115053934dcbf3d0a4b1a6427a1800c61ef6179e8dddd"} Jul 06 23:43:58.044098 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.044035 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nbl2d" podStartSLOduration=3.921344738 podStartE2EDuration="5.044014308s" totalImagesPullingTime="1.038398067s" totalInitContainerRuntime="84.271503ms" isStatefulPod=true podCreationTimestamp="2026-07-06 23:43:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:58.043829039 +0000 UTC m=+66.936017720" watchObservedRunningTime="2026-07-06 23:43:58.044014308 +0000 UTC m=+66.936202985" Jul 06 23:43:58.066100 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.066054 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-85b7659487-6965h"] Jul 06 23:43:58.066458 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.066416 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fp8mk" podStartSLOduration=3.759339106 podStartE2EDuration="5.066402255s" totalImagesPullingTime="1.307063149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:43:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:43:58.0651957 +0000 UTC m=+66.957384403" watchObservedRunningTime="2026-07-06 23:43:58.066402255 +0000 UTC m=+66.958590934" Jul 06 23:43:58.070681 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.070655 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.072986 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.072927 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 23:43:58.073125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.073103 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8hsrilacj25rf\"" Jul 06 23:43:58.073236 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.073223 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 23:43:58.073427 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.073412 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 23:43:58.073484 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.073473 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6xjb4\"" Jul 06 23:43:58.073632 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.073621 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 23:43:58.081636 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.081613 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85b7659487-6965h"] Jul 06 23:43:58.097214 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.097130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:58.100730 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.100698 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.100860 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.100844 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-audit-log\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.100923 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.100877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-secret-metrics-server-tls\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.100923 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.100904 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8fgp\" (UniqueName: \"kubernetes.io/projected/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-kube-api-access-b8fgp\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.101058 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.101028 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-secret-metrics-server-client-certs\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.101669 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.101640 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-client-ca-bundle\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.101799 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.101719 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-metrics-server-audit-profiles\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.161006 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.160974 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:58.203189 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.203136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-audit-log\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.203189 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.203180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-secret-metrics-server-tls\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.203433 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.203209 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8fgp\" (UniqueName: \"kubernetes.io/projected/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-kube-api-access-b8fgp\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.203433 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.203246 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-secret-metrics-server-client-certs\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.203433 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.203305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-client-ca-bundle\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.203433 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.203340 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-metrics-server-audit-profiles\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.203433 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.203386 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.203637 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.203604 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-audit-log\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.204146 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.204087 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.204541 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.204514 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-metrics-server-audit-profiles\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.206652 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.206614 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-secret-metrics-server-tls\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.206753 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.206727 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-secret-metrics-server-client-certs\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.215089 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.215036 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8fgp\" (UniqueName: \"kubernetes.io/projected/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-kube-api-access-b8fgp\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.218057 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.218032 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d607b3b8-b7c9-40d4-ad63-92d28e7c18b8-client-ca-bundle\") pod \"metrics-server-85b7659487-6965h\" (UID: \"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8\") " pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.260340 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:58.260308 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5222093_6d7d_46d1_93f1_47245a02ae3c.slice/crio-3c2ee2d1ed67bf5918f71ed82c16b65bf7cbbbc8edf3b3e4c75aafe8e7bf1a21 WatchSource:0}: Error finding container 3c2ee2d1ed67bf5918f71ed82c16b65bf7cbbbc8edf3b3e4c75aafe8e7bf1a21: Status 404 returned error can't find the container with id 3c2ee2d1ed67bf5918f71ed82c16b65bf7cbbbc8edf3b3e4c75aafe8e7bf1a21 Jul 06 23:43:58.383072 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.383031 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:43:58.518231 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.518155 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82"] Jul 06 23:43:58.521698 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.521538 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" Jul 06 23:43:58.524024 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.523972 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qrfl7\"" Jul 06 23:43:58.524369 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.524351 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 23:43:58.526134 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.526072 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:58.534176 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.534145 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82"] Jul 06 23:43:58.606147 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.606119 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85b7659487-6965h"] Jul 06 23:43:58.607329 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.607305 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26df84eb-c8c8-439a-91e8-25dc1a6e77e2-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-sbg82\" (UID: \"26df84eb-c8c8-439a-91e8-25dc1a6e77e2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" Jul 06 23:43:58.607417 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.607399 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdfsz\" (UniqueName: \"kubernetes.io/projected/26df84eb-c8c8-439a-91e8-25dc1a6e77e2-kube-api-access-kdfsz\") pod \"monitoring-plugin-d5dcff7cf-sbg82\" (UID: \"26df84eb-c8c8-439a-91e8-25dc1a6e77e2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" Jul 06 23:43:58.608357 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.608300 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:58.609650 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:43:58.609629 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd607b3b8_b7c9_40d4_ad63_92d28e7c18b8.slice/crio-1abf4ae597a36ce0a30d60570e76ccf30cf5aa41c65c49bfbd8ff3da31882a8e WatchSource:0}: Error finding container 1abf4ae597a36ce0a30d60570e76ccf30cf5aa41c65c49bfbd8ff3da31882a8e: Status 404 returned error can't find the container with id 1abf4ae597a36ce0a30d60570e76ccf30cf5aa41c65c49bfbd8ff3da31882a8e Jul 06 23:43:58.708502 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.708453 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26df84eb-c8c8-439a-91e8-25dc1a6e77e2-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-sbg82\" (UID: \"26df84eb-c8c8-439a-91e8-25dc1a6e77e2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" Jul 06 23:43:58.708906 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.708597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdfsz\" (UniqueName: \"kubernetes.io/projected/26df84eb-c8c8-439a-91e8-25dc1a6e77e2-kube-api-access-kdfsz\") pod \"monitoring-plugin-d5dcff7cf-sbg82\" (UID: \"26df84eb-c8c8-439a-91e8-25dc1a6e77e2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" Jul 06 23:43:58.709055 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:58.709037 2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 06 23:43:58.709110 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:43:58.709105 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26df84eb-c8c8-439a-91e8-25dc1a6e77e2-monitoring-plugin-cert podName:26df84eb-c8c8-439a-91e8-25dc1a6e77e2 nodeName:}" failed. No retries permitted until 2026-07-06 23:43:59.209084496 +0000 UTC m=+68.101273157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/26df84eb-c8c8-439a-91e8-25dc1a6e77e2-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-sbg82" (UID: "26df84eb-c8c8-439a-91e8-25dc1a6e77e2") : secret "monitoring-plugin-cert" not found Jul 06 23:43:58.718600 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:58.718510 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdfsz\" (UniqueName: \"kubernetes.io/projected/26df84eb-c8c8-439a-91e8-25dc1a6e77e2-kube-api-access-kdfsz\") pod \"monitoring-plugin-d5dcff7cf-sbg82\" (UID: \"26df84eb-c8c8-439a-91e8-25dc1a6e77e2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" Jul 06 23:43:59.014176 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.014054 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85b7659487-6965h" event={"ID":"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8","Type":"ContainerStarted","Data":"1abf4ae597a36ce0a30d60570e76ccf30cf5aa41c65c49bfbd8ff3da31882a8e"} Jul 06 23:43:59.017585 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.017398 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" event={"ID":"dec7b01b-e69f-423b-a48e-6919828a859f","Type":"ContainerStarted","Data":"26438eb9566f8609677325fb5ee205da4dc44d4c2fe10e80fd331a0b2126d460"} Jul 06 23:43:59.017585 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.017476 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" event={"ID":"dec7b01b-e69f-423b-a48e-6919828a859f","Type":"ContainerStarted","Data":"2f0869924c2108f40f781101fecfb6bd915bba4d90e8a68e204461fcd125d284"} Jul 06 23:43:59.017585 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.017519 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" event={"ID":"dec7b01b-e69f-423b-a48e-6919828a859f","Type":"ContainerStarted","Data":"044a294890b5528601ab3b48563da9913309062c9678943e658a1d8540776707"} Jul 06 23:43:59.020733 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.020695 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vlbd8" event={"ID":"d5222093-6d7d-46d1-93f1-47245a02ae3c","Type":"ContainerStarted","Data":"3c2ee2d1ed67bf5918f71ed82c16b65bf7cbbbc8edf3b3e4c75aafe8e7bf1a21"} Jul 06 23:43:59.197338 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.197311 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:59.215009 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.214976 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26df84eb-c8c8-439a-91e8-25dc1a6e77e2-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-sbg82\" (UID: \"26df84eb-c8c8-439a-91e8-25dc1a6e77e2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" Jul 06 23:43:59.225491 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.225431 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/26df84eb-c8c8-439a-91e8-25dc1a6e77e2-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-sbg82\" (UID: \"26df84eb-c8c8-439a-91e8-25dc1a6e77e2\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" Jul 06 23:43:59.405733 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.405693 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:59.439639 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.439384 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" Jul 06 23:43:59.511674 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.511643 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:59.616656 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.616557 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82"] Jul 06 23:43:59.726560 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.725634 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66c46df9c7-ft497"] Jul 06 23:43:59.731930 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.731866 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.740892 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.740771 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66c46df9c7-ft497"] Jul 06 23:43:59.820604 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.820484 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-serving-cert\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.820604 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.820539 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-trusted-ca-bundle\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.820828 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.820619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-config\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.820828 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.820653 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clb4c\" (UniqueName: \"kubernetes.io/projected/1c8f4079-d21c-4b74-b7f8-7aad419877e6-kube-api-access-clb4c\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.820828 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.820727 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-oauth-config\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.820828 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.820772 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-oauth-serving-cert\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.821021 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.820853 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-service-ca\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.922119 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.922082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-oauth-config\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.922344 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.922152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-oauth-serving-cert\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.922344 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.922193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-service-ca\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.922344 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.922239 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-serving-cert\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.922344 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.922267 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-trusted-ca-bundle\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.922344 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.922293 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-config\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.922344 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.922317 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clb4c\" (UniqueName: \"kubernetes.io/projected/1c8f4079-d21c-4b74-b7f8-7aad419877e6-kube-api-access-clb4c\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.923542 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.923493 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-config\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.925444 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.924314 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-trusted-ca-bundle\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.925444 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.924757 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-oauth-serving-cert\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.925444 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.925395 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-service-ca\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.926920 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.926634 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-oauth-config\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.927898 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.927872 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:43:59.934131 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.932079 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:43:59.934131 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.932322 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-serving-cert\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.938754 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.937632 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:43:59.938754 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.937888 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clb4c\" (UniqueName: \"kubernetes.io/projected/1c8f4079-d21c-4b74-b7f8-7aad419877e6-kube-api-access-clb4c\") pod \"console-66c46df9c7-ft497\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:43:59.940150 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.940129 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 23:43:59.940584 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.940552 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nvmz8\"" Jul 06 23:43:59.940866 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.940677 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 23:43:59.941285 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.941266 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 23:43:59.942214 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.941462 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 23:43:59.942214 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.941660 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 23:43:59.942214 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.941686 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 23:43:59.942214 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.941842 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 23:43:59.942507 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.942266 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 23:43:59.943378 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.942850 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 23:43:59.943378 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.942945 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b5olo1nidurl4\"" Jul 06 23:43:59.943378 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.943057 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 23:43:59.944649 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.944224 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 23:43:59.944649 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.944412 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 23:43:59.945845 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.945819 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 23:43:59.952248 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.952227 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 23:43:59.955167 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:43:59.955143 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:44:00.023070 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023070 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023079 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-web-config\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023070 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023102 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023125 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023150 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023177 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023209 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-config\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023232 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023262 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023290 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023317 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023693 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023352 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023693 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023392 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b39eccb2-5950-474e-ba58-bddbf5c16334-config-out\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023693 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023424 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b39eccb2-5950-474e-ba58-bddbf5c16334-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023693 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023448 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023693 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023472 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023693 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023501 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023693 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023527 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023693 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023559 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zmmt\" (UniqueName: \"kubernetes.io/projected/b39eccb2-5950-474e-ba58-bddbf5c16334-kube-api-access-4zmmt\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.023693 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.023612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.029315 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.029282 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f2vgw" event={"ID":"58cbc7f0-0b10-46b4-8f20-73034d089bc9","Type":"ContainerStarted","Data":"6ce386f2a8bff85a116b989583202f19d710df44983314caeb7eae73839f3d26"} Jul 06 23:44:00.029524 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.029348 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f2vgw" event={"ID":"58cbc7f0-0b10-46b4-8f20-73034d089bc9","Type":"ContainerStarted","Data":"56d9cd1f848b5b144dfc91beaccfdde3144fd64d5c095bcc6bf0c9d250512bcd"} Jul 06 23:44:00.046852 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.046818 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:44:00.047081 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.047036 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f2vgw" podStartSLOduration=67.542555344 podStartE2EDuration="1m9.047019827s" totalImagesPullingTime="1.504464483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:44:00.046003038 +0000 UTC m=+68.938191718" watchObservedRunningTime="2026-07-06 23:44:00.047019827 +0000 UTC m=+68.939208505" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.124378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-web-config\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.124427 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.124456 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.124932 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125110 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125155 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-config\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125250 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125307 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125346 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125383 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b39eccb2-5950-474e-ba58-bddbf5c16334-config-out\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125412 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b39eccb2-5950-474e-ba58-bddbf5c16334-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125436 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125461 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.125644 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125499 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.126601 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125524 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.126601 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125553 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zmmt\" (UniqueName: \"kubernetes.io/projected/b39eccb2-5950-474e-ba58-bddbf5c16334-kube-api-access-4zmmt\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.126601 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125609 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.126601 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.125723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.127742 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.126921 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.127742 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.127030 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.127742 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.127596 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.130586 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.130549 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.132483 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.130800 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.133436 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.133356 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.135989 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.134196 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b39eccb2-5950-474e-ba58-bddbf5c16334-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.135989 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.135100 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b39eccb2-5950-474e-ba58-bddbf5c16334-config-out\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.137410 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.137386 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-web-config\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.137798 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.137776 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.137798 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.137794 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.137942 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.137851 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-config\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.137942 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.137869 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.138063 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.138047 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.138353 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.138303 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.139302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.139279 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.140008 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.139923 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b39eccb2-5950-474e-ba58-bddbf5c16334-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.146216 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.146192 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zmmt\" (UniqueName: \"kubernetes.io/projected/b39eccb2-5950-474e-ba58-bddbf5c16334-kube-api-access-4zmmt\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.148790 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.148767 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b39eccb2-5950-474e-ba58-bddbf5c16334-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b39eccb2-5950-474e-ba58-bddbf5c16334\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.197907 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.197875 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:00.252893 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.252612 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:00.295336 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.295297 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:00.352607 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:44:00.347255 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26df84eb_c8c8_439a_91e8_25dc1a6e77e2.slice/crio-fb87c111591d05bd885d54523c6f8d3e3d0c550e34649ebcf37c104c525e3f91 WatchSource:0}: Error finding container fb87c111591d05bd885d54523c6f8d3e3d0c550e34649ebcf37c104c525e3f91: Status 404 returned error can't find the container with id fb87c111591d05bd885d54523c6f8d3e3d0c550e34649ebcf37c104c525e3f91 Jul 06 23:44:00.395456 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:00.395358 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:01.038316 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:01.038284 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" event={"ID":"26df84eb-c8c8-439a-91e8-25dc1a6e77e2","Type":"ContainerStarted","Data":"fb87c111591d05bd885d54523c6f8d3e3d0c550e34649ebcf37c104c525e3f91"} Jul 06 23:44:01.089276 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:01.089229 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66c46df9c7-ft497"] Jul 06 23:44:02.237688 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:44:02.237648 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8f4079_d21c_4b74_b7f8_7aad419877e6.slice/crio-0e46eca974fdd72faa62a8645cba0b3701b74aa504abbf440eb04167beca5af7 WatchSource:0}: Error finding container 0e46eca974fdd72faa62a8645cba0b3701b74aa504abbf440eb04167beca5af7: Status 404 returned error can't find the container with id 0e46eca974fdd72faa62a8645cba0b3701b74aa504abbf440eb04167beca5af7 Jul 06 23:44:02.384486 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:02.384451 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 23:44:03.047531 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:03.047491 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c46df9c7-ft497" event={"ID":"1c8f4079-d21c-4b74-b7f8-7aad419877e6","Type":"ContainerStarted","Data":"0e46eca974fdd72faa62a8645cba0b3701b74aa504abbf440eb04167beca5af7"} Jul 06 23:44:03.201556 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:03.201509 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:03.303169 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:03.303063 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:03.684589 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:44:03.684533 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39eccb2_5950_474e_ba58_bddbf5c16334.slice/crio-5986cff6d6ea311ff65c15ad3e01bba2a61e31a34772ff0ede27fe5645c784e9 WatchSource:0}: Error finding container 5986cff6d6ea311ff65c15ad3e01bba2a61e31a34772ff0ede27fe5645c784e9: Status 404 returned error can't find the container with id 5986cff6d6ea311ff65c15ad3e01bba2a61e31a34772ff0ede27fe5645c784e9 Jul 06 23:44:04.053679 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:04.053539 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b39eccb2-5950-474e-ba58-bddbf5c16334","Type":"ContainerStarted","Data":"5986cff6d6ea311ff65c15ad3e01bba2a61e31a34772ff0ede27fe5645c784e9"} Jul 06 23:44:04.282442 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:04.282381 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:05.061781 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.061739 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5","Type":"ContainerStarted","Data":"d8f594f7bffa5b7b1908f28170080386619735d4a4012b4ce3367ae4608041e3"} Jul 06 23:44:05.062265 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.061854 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5","Type":"ContainerStarted","Data":"0553a158aecaffa8092d8546821333b8f28879287e29fd0775a5912e937ee577"} Jul 06 23:44:05.062265 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.061898 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5","Type":"ContainerStarted","Data":"ee146d009aa2e505ee61d4acbdef15876e1f0144fff53c9a10fa11f0a8981fbc"} Jul 06 23:44:05.062265 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.061935 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5","Type":"ContainerStarted","Data":"ea9362d470f90e2bcdc98e7afe42c169ae1d5dcaf896d631beb48a52606687e0"} Jul 06 23:44:05.062265 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.061969 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5","Type":"ContainerStarted","Data":"e15d711df3322bc21bcc49bd380910b905641069f74bb21cb951d0b40a82f4df"} Jul 06 23:44:05.062265 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.062004 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d999c3e1-d0a7-4d69-96eb-f954ba0dafd5","Type":"ContainerStarted","Data":"2d2d0ff0e7d737a43ec3af7a52369ae9effb3625f50717541cb044deaf864db0"} Jul 06 23:44:05.063335 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.063302 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nng68" event={"ID":"2a524114-7358-43de-85d9-00e55e281c8a","Type":"ContainerStarted","Data":"f5aac7348cb24855c8e53493da909447a274f73996e0e1841180e14f88232b74"} Jul 06 23:44:05.064905 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.064869 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85b7659487-6965h" event={"ID":"d607b3b8-b7c9-40d4-ad63-92d28e7c18b8","Type":"ContainerStarted","Data":"acc7c130ab88b5d4e4dc8bc408352bb3f6fa58a773bdd14208d4a7842480a7d0"} Jul 06 23:44:05.066364 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.066337 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" event={"ID":"26df84eb-c8c8-439a-91e8-25dc1a6e77e2","Type":"ContainerStarted","Data":"32ee734d49d75d903d975ffdac08115e755ab5362ad1cabc2b8bc9902825a36c"} Jul 06 23:44:05.066530 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.066508 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" Jul 06 23:44:05.067992 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.067968 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c46df9c7-ft497" event={"ID":"1c8f4079-d21c-4b74-b7f8-7aad419877e6","Type":"ContainerStarted","Data":"0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7"} Jul 06 23:44:05.069635 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.069616 2560 generic.go:356] "Generic (PLEG): container finished" podID="b39eccb2-5950-474e-ba58-bddbf5c16334" containerID="9c34f3f839482f2ee9a18e7b46d9a47b4b7ec1fd0e5ed484d3b94c35aed1eed1" exitCode=0 Jul 06 23:44:05.069737 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.069687 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b39eccb2-5950-474e-ba58-bddbf5c16334","Type":"ContainerDied","Data":"9c34f3f839482f2ee9a18e7b46d9a47b4b7ec1fd0e5ed484d3b94c35aed1eed1"} Jul 06 23:44:05.072098 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.072079 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" Jul 06 23:44:05.073076 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.073054 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" event={"ID":"dec7b01b-e69f-423b-a48e-6919828a859f","Type":"ContainerStarted","Data":"f86580c3fdb0c540c626680a94b1a44b4db46a9e5b5eda575109362aa1ff8a9a"} Jul 06 23:44:05.073186 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.073117 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" event={"ID":"dec7b01b-e69f-423b-a48e-6919828a859f","Type":"ContainerStarted","Data":"dd9d0d2fabe5e60d33df56f31e641a95477608daa64f0b9f005a8a92c29ab8ff"} Jul 06 23:44:05.073186 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.073159 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" event={"ID":"dec7b01b-e69f-423b-a48e-6919828a859f","Type":"ContainerStarted","Data":"40d3c4f025232ae768d568ce24bcfaf1fa259ea5fea85b59291cbe94d3b7269d"} Jul 06 23:44:05.073298 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.073201 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:44:05.074788 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.074766 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vlbd8" event={"ID":"d5222093-6d7d-46d1-93f1-47245a02ae3c","Type":"ContainerStarted","Data":"bc852401912284d54dfc7579fcdc7cfcd3eb8b253947f08c6b6725eed006e34c"} Jul 06 23:44:05.074914 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.074899 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:44:05.078438 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.078421 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" Jul 06 23:44:05.089049 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.089006 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.481303841 podStartE2EDuration="11.088987356s" totalImagesPullingTime="7.579157997s" totalInitContainerRuntime="28.525518ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:43:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:44:05.087702634 +0000 UTC m=+73.979891312" watchObservedRunningTime="2026-07-06 23:44:05.088987356 +0000 UTC m=+73.981176036" Jul 06 23:44:05.113223 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.113155 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7b77bb6784-dwf8f" podStartSLOduration=2.902493617 podStartE2EDuration="10.113132636s" totalImagesPullingTime="7.210639019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:43:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:44:05.111380284 +0000 UTC m=+74.003568965" watchObservedRunningTime="2026-07-06 23:44:05.113132636 +0000 UTC m=+74.005321315" Jul 06 23:44:05.128289 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.128237 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-85b7659487-6965h" podStartSLOduration=1.697276448 podStartE2EDuration="7.128221642s" totalImagesPullingTime="5.430945194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:43:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:44:05.12733213 +0000 UTC m=+74.019520808" watchObservedRunningTime="2026-07-06 23:44:05.128221642 +0000 UTC m=+74.020410320" Jul 06 23:44:05.148330 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.148278 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66c46df9c7-ft497" podStartSLOduration=6.148259129 podStartE2EDuration="6.148259129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:43:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:44:05.1468263 +0000 UTC m=+74.039014979" watchObservedRunningTime="2026-07-06 23:44:05.148259129 +0000 UTC m=+74.040447807" Jul 06 23:44:05.187339 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.187292 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-sbg82" podStartSLOduration=3.426658149 podStartE2EDuration="7.187275335s" totalImagesPullingTime="3.760617186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:43:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:44:05.186710865 +0000 UTC m=+74.078899542" watchObservedRunningTime="2026-07-06 23:44:05.187275335 +0000 UTC m=+74.079464013" Jul 06 23:44:05.200858 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.200592 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nng68" podStartSLOduration=66.835840128 podStartE2EDuration="1m13.200548593s" totalImagesPullingTime="6.364708465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:44:05.200353036 +0000 UTC m=+74.092541711" watchObservedRunningTime="2026-07-06 23:44:05.200548593 +0000 UTC m=+74.092737270" Jul 06 23:44:05.215667 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:05.215615 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vlbd8" podStartSLOduration=67.433605588 podStartE2EDuration="1m13.215597402s" totalImagesPullingTime="5.781991814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:44:05.21466697 +0000 UTC m=+74.106855649" watchObservedRunningTime="2026-07-06 23:44:05.215597402 +0000 UTC m=+74.107786077" Jul 06 23:44:08.478514 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:08.478482 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:08.573775 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:08.573746 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:09.097744 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:09.097713 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b39eccb2-5950-474e-ba58-bddbf5c16334","Type":"ContainerStarted","Data":"ce1c93f3c269614139d2a5f4337a7d5d7fae219b7d502b6ddcbf28289a99c083"} Jul 06 23:44:09.097904 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:09.097800 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b39eccb2-5950-474e-ba58-bddbf5c16334","Type":"ContainerStarted","Data":"b2f40fd89ca61f042de29da7f22b822b60cd2548270277d24a6ff9571a7b2c07"} Jul 06 23:44:09.097904 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:09.097827 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b39eccb2-5950-474e-ba58-bddbf5c16334","Type":"ContainerStarted","Data":"7d965babb1e197e99eeca7bdd08a3e2bf8361ec6d4b8c719e01a818e1a83905f"} Jul 06 23:44:09.097904 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:09.097851 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b39eccb2-5950-474e-ba58-bddbf5c16334","Type":"ContainerStarted","Data":"84d1a3938875eabe40270b0f73f51bed096bb02ec6788a1f7564698c9536e42a"} Jul 06 23:44:09.097904 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:09.097874 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b39eccb2-5950-474e-ba58-bddbf5c16334","Type":"ContainerStarted","Data":"8dc80f8d930796e941017f6a557dc0fcd2846131be32dbcf63c730690d91823c"} Jul 06 23:44:09.097904 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:09.097895 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b39eccb2-5950-474e-ba58-bddbf5c16334","Type":"ContainerStarted","Data":"b7763b105f8573222eb3bcc9f8829e0060d017236e8191c1a5ec417e9b6f75a5"} Jul 06 23:44:09.125863 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:09.125790 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.011422812 podStartE2EDuration="10.125769321s" totalImagesPullingTime="3.067237106s" totalInitContainerRuntime="47.109403ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:44:09.12455459 +0000 UTC m=+78.016743272" watchObservedRunningTime="2026-07-06 23:44:09.125769321 +0000 UTC m=+78.017958003" Jul 06 23:44:10.047943 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:10.047906 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:44:10.048399 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:10.047969 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:44:10.052641 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:10.052622 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:44:10.105383 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:10.105359 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:44:10.152897 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:10.152868 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8bf8c8f46-tvptr"] Jul 06 23:44:10.253150 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:10.253122 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:44:10.323614 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:10.323520 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:10.423259 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:10.423226 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:12.274846 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:12.274737 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:12.374049 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:12.374015 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:18.383224 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:18.383189 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:44:18.383654 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:18.383244 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:44:23.499314 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:23.499282 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:23.590847 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:23.590806 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:34.158979 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:34.158941 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:35.174170 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:35.174112 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8bf8c8f46-tvptr" podUID="0e736abd-483f-4266-a13f-ba1d815ea7d6" containerName="console" containerID="cri-o://6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598" gracePeriod=15 Jul 06 23:44:35.947801 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:35.947770 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:44:36.083302 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.083239 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vlbd8" Jul 06 23:44:36.132344 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.132319 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-config\") pod \"0e736abd-483f-4266-a13f-ba1d815ea7d6\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " Jul 06 23:44:36.132482 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.132365 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-trusted-ca-bundle\") pod \"0e736abd-483f-4266-a13f-ba1d815ea7d6\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " Jul 06 23:44:36.132482 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.132393 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-oauth-config\") pod \"0e736abd-483f-4266-a13f-ba1d815ea7d6\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " Jul 06 23:44:36.132482 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.132415 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-service-ca\") pod \"0e736abd-483f-4266-a13f-ba1d815ea7d6\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " Jul 06 23:44:36.133082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.132666 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-oauth-serving-cert\") pod \"0e736abd-483f-4266-a13f-ba1d815ea7d6\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " Jul 06 23:44:36.133082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.132761 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wg88\" (UniqueName: \"kubernetes.io/projected/0e736abd-483f-4266-a13f-ba1d815ea7d6-kube-api-access-6wg88\") pod \"0e736abd-483f-4266-a13f-ba1d815ea7d6\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " Jul 06 23:44:36.133082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.132767 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-config" (OuterVolumeSpecName: "console-config") pod "0e736abd-483f-4266-a13f-ba1d815ea7d6" (UID: "0e736abd-483f-4266-a13f-ba1d815ea7d6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:44:36.133082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.132806 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-serving-cert\") pod \"0e736abd-483f-4266-a13f-ba1d815ea7d6\" (UID: \"0e736abd-483f-4266-a13f-ba1d815ea7d6\") " Jul 06 23:44:36.133082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.132813 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-service-ca" (OuterVolumeSpecName: "service-ca") pod "0e736abd-483f-4266-a13f-ba1d815ea7d6" (UID: "0e736abd-483f-4266-a13f-ba1d815ea7d6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:44:36.133082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.132844 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0e736abd-483f-4266-a13f-ba1d815ea7d6" (UID: "0e736abd-483f-4266-a13f-ba1d815ea7d6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:44:36.133082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.133027 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-config\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:44:36.133082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.133045 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-trusted-ca-bundle\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:44:36.133082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.133039 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0e736abd-483f-4266-a13f-ba1d815ea7d6" (UID: "0e736abd-483f-4266-a13f-ba1d815ea7d6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:44:36.133082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.133059 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-service-ca\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:44:36.135141 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.135105 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0e736abd-483f-4266-a13f-ba1d815ea7d6" (UID: "0e736abd-483f-4266-a13f-ba1d815ea7d6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:44:36.135254 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.135152 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e736abd-483f-4266-a13f-ba1d815ea7d6-kube-api-access-6wg88" (OuterVolumeSpecName: "kube-api-access-6wg88") pod "0e736abd-483f-4266-a13f-ba1d815ea7d6" (UID: "0e736abd-483f-4266-a13f-ba1d815ea7d6"). InnerVolumeSpecName "kube-api-access-6wg88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:44:36.135254 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.135159 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0e736abd-483f-4266-a13f-ba1d815ea7d6" (UID: "0e736abd-483f-4266-a13f-ba1d815ea7d6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:44:36.206475 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.206450 2560 generic.go:356] "Generic (PLEG): container finished" podID="0e736abd-483f-4266-a13f-ba1d815ea7d6" containerID="6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598" exitCode=0 Jul 06 23:44:36.206779 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.206491 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bf8c8f46-tvptr" Jul 06 23:44:36.206779 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.206519 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bf8c8f46-tvptr" event={"ID":"0e736abd-483f-4266-a13f-ba1d815ea7d6","Type":"ContainerDied","Data":"6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598"} Jul 06 23:44:36.206779 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.206557 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bf8c8f46-tvptr" event={"ID":"0e736abd-483f-4266-a13f-ba1d815ea7d6","Type":"ContainerDied","Data":"35880a46be07e419f9c7b25d212754b21596b1a1972530f7ed183c153013dbbb"} Jul 06 23:44:36.206779 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.206600 2560 scope.go:117] "RemoveContainer" containerID="6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598" Jul 06 23:44:36.218248 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.218221 2560 scope.go:117] "RemoveContainer" containerID="6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598" Jul 06 23:44:36.218479 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:44:36.218460 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598\": container with ID starting with 6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598 not found: ID does not exist" containerID="6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598" Jul 06 23:44:36.218543 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.218488 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598"} err="failed to get container status \"6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598\": rpc error: code = NotFound desc = could not find container \"6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598\": container with ID starting with 6b28366ba3b9e1363132a719c1a332e350b9bff79825004c3a9d0fefc4d69598 not found: ID does not exist" Jul 06 23:44:36.229335 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.229313 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8bf8c8f46-tvptr"] Jul 06 23:44:36.233052 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.233032 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8bf8c8f46-tvptr"] Jul 06 23:44:36.233882 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.233863 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wg88\" (UniqueName: \"kubernetes.io/projected/0e736abd-483f-4266-a13f-ba1d815ea7d6-kube-api-access-6wg88\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:44:36.233882 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.233882 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-serving-cert\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:44:36.234000 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.233892 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e736abd-483f-4266-a13f-ba1d815ea7d6-console-oauth-config\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:44:36.234000 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:36.233900 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e736abd-483f-4266-a13f-ba1d815ea7d6-oauth-serving-cert\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:44:37.561628 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:37.561599 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e736abd-483f-4266-a13f-ba1d815ea7d6" path="/var/lib/kubelet/pods/0e736abd-483f-4266-a13f-ba1d815ea7d6/volumes" Jul 06 23:44:38.389626 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:38.389596 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:44:38.393477 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:38.393450 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-85b7659487-6965h" Jul 06 23:44:38.458746 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:38.458711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:44:38.516920 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:44:38.516879 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:00.253594 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:00.253541 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:45:00.274090 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:00.274060 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:45:00.328818 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:00.328790 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 23:45:12.950779 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:12.950737 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:13.051615 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:13.051552 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:23.504535 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:23.504496 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:23.613380 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:23.613346 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:23.661244 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:23.661208 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:24.224127 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:24.224095 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:24.275123 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:24.275087 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:28.178512 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:28.178476 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:28.280046 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:28.280004 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:31.977138 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:31.977101 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:39.671878 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:39.671828 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66c46df9c7-ft497"] Jul 06 23:45:39.777191 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:39.777151 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:39.878897 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:39.878847 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:39.935687 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:39.935560 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:40.052578 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:40.052548 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:40.154254 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:40.154203 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:47.414728 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:47.414649 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:47.515682 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:47.515644 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:53.472121 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:53.472082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:45:53.573064 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:45:53.573031 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:46:04.693751 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:04.693683 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-66c46df9c7-ft497" podUID="1c8f4079-d21c-4b74-b7f8-7aad419877e6" containerName="console" containerID="cri-o://0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7" gracePeriod=15 Jul 06 23:46:04.951370 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:04.951347 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:46:05.128109 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.128075 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-oauth-config\") pod \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " Jul 06 23:46:05.128312 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.128160 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-trusted-ca-bundle\") pod \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " Jul 06 23:46:05.128312 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.128209 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clb4c\" (UniqueName: \"kubernetes.io/projected/1c8f4079-d21c-4b74-b7f8-7aad419877e6-kube-api-access-clb4c\") pod \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " Jul 06 23:46:05.128312 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.128241 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-config\") pod \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " Jul 06 23:46:05.128312 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.128283 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-serving-cert\") pod \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " Jul 06 23:46:05.128312 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.128307 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-oauth-serving-cert\") pod \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " Jul 06 23:46:05.128555 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.128376 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-service-ca\") pod \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\" (UID: \"1c8f4079-d21c-4b74-b7f8-7aad419877e6\") " Jul 06 23:46:05.128779 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.128745 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1c8f4079-d21c-4b74-b7f8-7aad419877e6" (UID: "1c8f4079-d21c-4b74-b7f8-7aad419877e6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:46:05.128953 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.128928 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-service-ca" (OuterVolumeSpecName: "service-ca") pod "1c8f4079-d21c-4b74-b7f8-7aad419877e6" (UID: "1c8f4079-d21c-4b74-b7f8-7aad419877e6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:46:05.129082 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.128906 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1c8f4079-d21c-4b74-b7f8-7aad419877e6" (UID: "1c8f4079-d21c-4b74-b7f8-7aad419877e6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:46:05.129139 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.129067 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-config" (OuterVolumeSpecName: "console-config") pod "1c8f4079-d21c-4b74-b7f8-7aad419877e6" (UID: "1c8f4079-d21c-4b74-b7f8-7aad419877e6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:46:05.130649 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.130607 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1c8f4079-d21c-4b74-b7f8-7aad419877e6" (UID: "1c8f4079-d21c-4b74-b7f8-7aad419877e6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:46:05.130753 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.130701 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c8f4079-d21c-4b74-b7f8-7aad419877e6-kube-api-access-clb4c" (OuterVolumeSpecName: "kube-api-access-clb4c") pod "1c8f4079-d21c-4b74-b7f8-7aad419877e6" (UID: "1c8f4079-d21c-4b74-b7f8-7aad419877e6"). InnerVolumeSpecName "kube-api-access-clb4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:46:05.130753 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.130717 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1c8f4079-d21c-4b74-b7f8-7aad419877e6" (UID: "1c8f4079-d21c-4b74-b7f8-7aad419877e6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:46:05.229289 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.229185 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-serving-cert\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:46:05.229289 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.229221 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-oauth-serving-cert\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:46:05.229289 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.229232 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-service-ca\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:46:05.229289 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.229240 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-oauth-config\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:46:05.229289 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.229250 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-trusted-ca-bundle\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:46:05.229289 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.229258 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clb4c\" (UniqueName: \"kubernetes.io/projected/1c8f4079-d21c-4b74-b7f8-7aad419877e6-kube-api-access-clb4c\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:46:05.229289 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.229267 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c8f4079-d21c-4b74-b7f8-7aad419877e6-console-config\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:46:05.563617 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.563496 2560 generic.go:356] "Generic (PLEG): container finished" podID="1c8f4079-d21c-4b74-b7f8-7aad419877e6" containerID="0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7" exitCode=0 Jul 06 23:46:05.563617 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.563544 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c46df9c7-ft497" event={"ID":"1c8f4079-d21c-4b74-b7f8-7aad419877e6","Type":"ContainerDied","Data":"0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7"} Jul 06 23:46:05.563617 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.563554 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c46df9c7-ft497" Jul 06 23:46:05.563617 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.563602 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c46df9c7-ft497" event={"ID":"1c8f4079-d21c-4b74-b7f8-7aad419877e6","Type":"ContainerDied","Data":"0e46eca974fdd72faa62a8645cba0b3701b74aa504abbf440eb04167beca5af7"} Jul 06 23:46:05.563617 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.563623 2560 scope.go:117] "RemoveContainer" containerID="0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7" Jul 06 23:46:05.573262 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.573243 2560 scope.go:117] "RemoveContainer" containerID="0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7" Jul 06 23:46:05.573561 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:46:05.573543 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7\": container with ID starting with 0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7 not found: ID does not exist" containerID="0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7" Jul 06 23:46:05.573639 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.573589 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7"} err="failed to get container status \"0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7\": rpc error: code = NotFound desc = could not find container \"0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7\": container with ID starting with 0a83d16f87234f6d840f71042efe8c387bb05e26826149dd09b70e2261b9ffc7 not found: ID does not exist" Jul 06 23:46:05.584630 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.584599 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66c46df9c7-ft497"] Jul 06 23:46:05.588388 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:05.588362 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66c46df9c7-ft497"] Jul 06 23:46:07.562045 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:07.562009 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c8f4079-d21c-4b74-b7f8-7aad419877e6" path="/var/lib/kubelet/pods/1c8f4079-d21c-4b74-b7f8-7aad419877e6/volumes" Jul 06 23:46:41.383919 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:41.383887 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:46:41.479241 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:46:41.479199 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:02.996655 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:02.996621 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:03.982478 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:03.982444 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:06.607221 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:06.607184 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:07.956020 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:07.955982 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:08.386790 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:08.386696 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:08.623509 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:08.623455 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:08.909688 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:08.909654 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:09.006739 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:09.006701 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:11.271132 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.271100 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:11.439975 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.439943 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:11.514051 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.514012 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-77vqv"] Jul 06 23:47:11.514365 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.514353 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e736abd-483f-4266-a13f-ba1d815ea7d6" containerName="console" Jul 06 23:47:11.514407 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.514367 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e736abd-483f-4266-a13f-ba1d815ea7d6" containerName="console" Jul 06 23:47:11.514407 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.514385 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c8f4079-d21c-4b74-b7f8-7aad419877e6" containerName="console" Jul 06 23:47:11.514407 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.514391 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c8f4079-d21c-4b74-b7f8-7aad419877e6" containerName="console" Jul 06 23:47:11.514503 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.514443 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c8f4079-d21c-4b74-b7f8-7aad419877e6" containerName="console" Jul 06 23:47:11.514503 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.514454 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e736abd-483f-4266-a13f-ba1d815ea7d6" containerName="console" Jul 06 23:47:11.517338 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.517321 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-77vqv" Jul 06 23:47:11.519415 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.519386 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 23:47:11.519591 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.519443 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 23:47:11.520314 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.520300 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-tnmgt\"" Jul 06 23:47:11.523715 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.523603 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-77vqv"] Jul 06 23:47:11.679084 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.679044 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8d5h\" (UniqueName: \"kubernetes.io/projected/e1a84a55-8eaa-4277-8448-07bc60018739-kube-api-access-p8d5h\") pod \"cert-manager-cainjector-559cdd8c87-77vqv\" (UID: \"e1a84a55-8eaa-4277-8448-07bc60018739\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-77vqv" Jul 06 23:47:11.679273 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.679103 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1a84a55-8eaa-4277-8448-07bc60018739-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-77vqv\" (UID: \"e1a84a55-8eaa-4277-8448-07bc60018739\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-77vqv" Jul 06 23:47:11.779634 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.779477 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8d5h\" (UniqueName: \"kubernetes.io/projected/e1a84a55-8eaa-4277-8448-07bc60018739-kube-api-access-p8d5h\") pod \"cert-manager-cainjector-559cdd8c87-77vqv\" (UID: \"e1a84a55-8eaa-4277-8448-07bc60018739\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-77vqv" Jul 06 23:47:11.779634 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.779545 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1a84a55-8eaa-4277-8448-07bc60018739-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-77vqv\" (UID: \"e1a84a55-8eaa-4277-8448-07bc60018739\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-77vqv" Jul 06 23:47:11.791707 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.791682 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8d5h\" (UniqueName: \"kubernetes.io/projected/e1a84a55-8eaa-4277-8448-07bc60018739-kube-api-access-p8d5h\") pod \"cert-manager-cainjector-559cdd8c87-77vqv\" (UID: \"e1a84a55-8eaa-4277-8448-07bc60018739\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-77vqv" Jul 06 23:47:11.798711 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.798683 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1a84a55-8eaa-4277-8448-07bc60018739-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-77vqv\" (UID: \"e1a84a55-8eaa-4277-8448-07bc60018739\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-77vqv" Jul 06 23:47:11.836997 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.836955 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-77vqv" Jul 06 23:47:11.971413 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:11.971383 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-77vqv"] Jul 06 23:47:11.973768 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:47:11.973725 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1a84a55_8eaa_4277_8448_07bc60018739.slice/crio-4033f9993e46f857f7cdce0e5a9f3b98e1b76b7e04f6bd9aa666d7b601af3ecb WatchSource:0}: Error finding container 4033f9993e46f857f7cdce0e5a9f3b98e1b76b7e04f6bd9aa666d7b601af3ecb: Status 404 returned error can't find the container with id 4033f9993e46f857f7cdce0e5a9f3b98e1b76b7e04f6bd9aa666d7b601af3ecb Jul 06 23:47:12.820120 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:12.820083 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-77vqv" event={"ID":"e1a84a55-8eaa-4277-8448-07bc60018739","Type":"ContainerStarted","Data":"4033f9993e46f857f7cdce0e5a9f3b98e1b76b7e04f6bd9aa666d7b601af3ecb"} Jul 06 23:47:13.597739 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:13.597701 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:13.652520 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:13.652479 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:15.835193 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:15.835156 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-77vqv" event={"ID":"e1a84a55-8eaa-4277-8448-07bc60018739","Type":"ContainerStarted","Data":"f9459f1022d91b3616937c0fce22a276dd1fcd53d5dbc4d3ad7b3a33ba1db31c"} Jul 06 23:47:15.852732 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:15.852679 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-77vqv" podStartSLOduration=1.566736552 podStartE2EDuration="4.852657761s" totalImagesPullingTime="3.285921209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:47:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:47:15.852080797 +0000 UTC m=+264.744269475" watchObservedRunningTime="2026-07-06 23:47:15.852657761 +0000 UTC m=+264.744846440" Jul 06 23:47:16.076459 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:16.076423 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:16.143939 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:16.143856 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:18.850875 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:18.850840 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:21.307741 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:21.307704 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:21.402269 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:21.402233 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:22.730021 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.729979 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9"] Jul 06 23:47:22.733531 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.733505 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9" Jul 06 23:47:22.736016 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.735989 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 23:47:22.736016 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.736009 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:47:22.736711 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.736697 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-dvhjz\"" Jul 06 23:47:22.741352 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.741331 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9"] Jul 06 23:47:22.768897 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.768853 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz7cd\" (UniqueName: \"kubernetes.io/projected/fb09c2ac-d728-4403-bec3-419aecd38502-kube-api-access-vz7cd\") pod \"openshift-lws-operator-fd8ccff4c-hdlf9\" (UID: \"fb09c2ac-d728-4403-bec3-419aecd38502\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9" Jul 06 23:47:22.769080 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.768927 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb09c2ac-d728-4403-bec3-419aecd38502-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hdlf9\" (UID: \"fb09c2ac-d728-4403-bec3-419aecd38502\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9" Jul 06 23:47:22.869635 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.869595 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb09c2ac-d728-4403-bec3-419aecd38502-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hdlf9\" (UID: \"fb09c2ac-d728-4403-bec3-419aecd38502\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9" Jul 06 23:47:22.869834 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.869665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz7cd\" (UniqueName: \"kubernetes.io/projected/fb09c2ac-d728-4403-bec3-419aecd38502-kube-api-access-vz7cd\") pod \"openshift-lws-operator-fd8ccff4c-hdlf9\" (UID: \"fb09c2ac-d728-4403-bec3-419aecd38502\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9" Jul 06 23:47:22.869999 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.869981 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb09c2ac-d728-4403-bec3-419aecd38502-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hdlf9\" (UID: \"fb09c2ac-d728-4403-bec3-419aecd38502\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9" Jul 06 23:47:22.878684 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:22.878648 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz7cd\" (UniqueName: \"kubernetes.io/projected/fb09c2ac-d728-4403-bec3-419aecd38502-kube-api-access-vz7cd\") pod \"openshift-lws-operator-fd8ccff4c-hdlf9\" (UID: \"fb09c2ac-d728-4403-bec3-419aecd38502\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9" Jul 06 23:47:23.044345 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:23.044260 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9" Jul 06 23:47:23.180063 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:23.180034 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9"] Jul 06 23:47:23.183039 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:47:23.183010 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb09c2ac_d728_4403_bec3_419aecd38502.slice/crio-eef42ebd4c36f64641ceb83830691658f19c1e399c06e96164a6c9fdbe07503a WatchSource:0}: Error finding container eef42ebd4c36f64641ceb83830691658f19c1e399c06e96164a6c9fdbe07503a: Status 404 returned error can't find the container with id eef42ebd4c36f64641ceb83830691658f19c1e399c06e96164a6c9fdbe07503a Jul 06 23:47:23.868740 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:23.868697 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9" event={"ID":"fb09c2ac-d728-4403-bec3-419aecd38502","Type":"ContainerStarted","Data":"eef42ebd4c36f64641ceb83830691658f19c1e399c06e96164a6c9fdbe07503a"} Jul 06 23:47:25.879472 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:25.879436 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9" event={"ID":"fb09c2ac-d728-4403-bec3-419aecd38502","Type":"ContainerStarted","Data":"7f5315cefec295c57f8d20cc1a9c72dac807f731095f5159d521bd26be2aa405"} Jul 06 23:47:25.896223 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:25.896163 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hdlf9" podStartSLOduration=1.42380626 podStartE2EDuration="3.896142783s" totalImagesPullingTime="2.472336523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:47:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:47:25.894345286 +0000 UTC m=+274.786533965" watchObservedRunningTime="2026-07-06 23:47:25.896142783 +0000 UTC m=+274.788331460" Jul 06 23:47:26.616928 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:26.616896 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:26.713333 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:26.713294 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:29.161110 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:29.161075 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:44.468233 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:44.468199 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:44.576458 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:44.576419 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:45.049399 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.049358 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:45.347584 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.347473 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:45.443618 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.443583 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:45.503980 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.503947 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv"] Jul 06 23:47:45.507424 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.507399 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.510429 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.510407 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 23:47:45.510814 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.510785 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 23:47:45.510814 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.510815 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 23:47:45.511013 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.510856 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 23:47:45.511013 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.510815 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-clcwx\"" Jul 06 23:47:45.530696 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.530663 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv"] Jul 06 23:47:45.564426 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.564383 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-tmp\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.564426 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.564423 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.564725 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.564537 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlnhm\" (UniqueName: \"kubernetes.io/projected/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-kube-api-access-dlnhm\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.564725 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.564633 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-manifests\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.564725 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.564676 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.665935 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.665835 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlnhm\" (UniqueName: \"kubernetes.io/projected/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-kube-api-access-dlnhm\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.665935 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.665898 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-manifests\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.665935 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.665931 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.666248 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.665957 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-tmp\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.666248 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.665976 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.666527 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.666502 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-manifests\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.668632 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.668606 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-tmp\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.668795 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.668777 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.668841 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.668810 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.676912 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.676883 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlnhm\" (UniqueName: \"kubernetes.io/projected/b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a-kube-api-access-dlnhm\") pod \"opendatahub-operator-controller-manager-5bfc4f9689-jcqlv\" (UID: \"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a\") " pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.746559 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.746522 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:45.818558 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.818515 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:45.854122 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.854088 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:45.926024 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.925843 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:45.964281 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.964257 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv"] Jul 06 23:47:45.967211 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:47:45.967181 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8bf98f3_eb4a_4bb3_af21_e57da6d7bf0a.slice/crio-968c2d0f403298cf5568ff9c0a8ce274275f51bd97c26ec7122062f3d2525f2b WatchSource:0}: Error finding container 968c2d0f403298cf5568ff9c0a8ce274275f51bd97c26ec7122062f3d2525f2b: Status 404 returned error can't find the container with id 968c2d0f403298cf5568ff9c0a8ce274275f51bd97c26ec7122062f3d2525f2b Jul 06 23:47:45.988159 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:45.988131 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:46.086837 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:46.086802 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:46.188038 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:46.187954 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:46.292033 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:46.291995 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:46.964001 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:46.963966 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" event={"ID":"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a","Type":"ContainerStarted","Data":"968c2d0f403298cf5568ff9c0a8ce274275f51bd97c26ec7122062f3d2525f2b"} Jul 06 23:47:49.982078 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:49.982040 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" event={"ID":"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a","Type":"ContainerStarted","Data":"0647385f57b248894efb222fc6d60db9fc2a6402ec4a1fd1d000e40f3dc31267"} Jul 06 23:47:50.988461 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:50.988424 2560 generic.go:356] "Generic (PLEG): container finished" podID="b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a" containerID="0647385f57b248894efb222fc6d60db9fc2a6402ec4a1fd1d000e40f3dc31267" exitCode=0 Jul 06 23:47:50.988882 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:50.988480 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" event={"ID":"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a","Type":"ContainerDied","Data":"0647385f57b248894efb222fc6d60db9fc2a6402ec4a1fd1d000e40f3dc31267"} Jul 06 23:47:51.478966 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:51.478933 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 23:47:51.994452 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:51.994412 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" event={"ID":"b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a","Type":"ContainerStarted","Data":"5f4e68e4f0c709944fe6802a0ee12dffc23b7098d945b20a6064d85eb3d0c387"} Jul 06 23:47:51.996958 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:51.994553 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:47:52.025828 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:52.025766 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" podStartSLOduration=2.276112448 podStartE2EDuration="7.025746354s" totalImagesPullingTime="3.59239554s" totalInitContainerRuntime="1.157238366s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:47:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:47:52.024045223 +0000 UTC m=+300.916233902" watchObservedRunningTime="2026-07-06 23:47:52.025746354 +0000 UTC m=+300.917935094" Jul 06 23:47:55.403141 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:55.403075 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:47:55.498038 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:47:55.498003 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:03.001438 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:03.001406 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5bfc4f9689-jcqlv" Jul 06 23:48:04.593885 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:04.593848 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:04.710747 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:04.710710 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:05.056142 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.056097 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:05.168910 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.168874 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q"] Jul 06 23:48:05.178117 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.178088 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" Jul 06 23:48:05.180794 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.180771 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 23:48:05.181760 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.181731 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-s27px\"" Jul 06 23:48:05.181760 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.181731 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 23:48:05.181897 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.181740 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 23:48:05.182340 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.182322 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 23:48:05.189694 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.189670 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q"] Jul 06 23:48:05.313430 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.313321 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:05.341249 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.341212 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5dc851-5a03-4587-8f36-f5408da0a3c2-tls-certs\") pod \"kube-auth-proxy-5646546ff9-ctv9q\" (UID: \"2d5dc851-5a03-4587-8f36-f5408da0a3c2\") " pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" Jul 06 23:48:05.341450 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.341326 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d5dc851-5a03-4587-8f36-f5408da0a3c2-tmp\") pod \"kube-auth-proxy-5646546ff9-ctv9q\" (UID: \"2d5dc851-5a03-4587-8f36-f5408da0a3c2\") " pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" Jul 06 23:48:05.341450 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.341355 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9fv5\" (UniqueName: \"kubernetes.io/projected/2d5dc851-5a03-4587-8f36-f5408da0a3c2-kube-api-access-w9fv5\") pod \"kube-auth-proxy-5646546ff9-ctv9q\" (UID: \"2d5dc851-5a03-4587-8f36-f5408da0a3c2\") " pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" Jul 06 23:48:05.412249 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.412217 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:05.442413 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.442380 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d5dc851-5a03-4587-8f36-f5408da0a3c2-tmp\") pod \"kube-auth-proxy-5646546ff9-ctv9q\" (UID: \"2d5dc851-5a03-4587-8f36-f5408da0a3c2\") " pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" Jul 06 23:48:05.442622 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.442421 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9fv5\" (UniqueName: \"kubernetes.io/projected/2d5dc851-5a03-4587-8f36-f5408da0a3c2-kube-api-access-w9fv5\") pod \"kube-auth-proxy-5646546ff9-ctv9q\" (UID: \"2d5dc851-5a03-4587-8f36-f5408da0a3c2\") " pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" Jul 06 23:48:05.442622 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.442459 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5dc851-5a03-4587-8f36-f5408da0a3c2-tls-certs\") pod \"kube-auth-proxy-5646546ff9-ctv9q\" (UID: \"2d5dc851-5a03-4587-8f36-f5408da0a3c2\") " pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" Jul 06 23:48:05.444919 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.444894 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d5dc851-5a03-4587-8f36-f5408da0a3c2-tmp\") pod \"kube-auth-proxy-5646546ff9-ctv9q\" (UID: \"2d5dc851-5a03-4587-8f36-f5408da0a3c2\") " pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" Jul 06 23:48:05.445050 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.445001 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5dc851-5a03-4587-8f36-f5408da0a3c2-tls-certs\") pod \"kube-auth-proxy-5646546ff9-ctv9q\" (UID: \"2d5dc851-5a03-4587-8f36-f5408da0a3c2\") " pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" Jul 06 23:48:05.454589 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.454543 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9fv5\" (UniqueName: \"kubernetes.io/projected/2d5dc851-5a03-4587-8f36-f5408da0a3c2-kube-api-access-w9fv5\") pod \"kube-auth-proxy-5646546ff9-ctv9q\" (UID: \"2d5dc851-5a03-4587-8f36-f5408da0a3c2\") " pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" Jul 06 23:48:05.488414 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.488350 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" Jul 06 23:48:05.509223 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.509185 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:05.609149 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.609108 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:05.632015 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.631978 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q"] Jul 06 23:48:05.635323 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:48:05.635302 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d5dc851_5a03_4587_8f36_f5408da0a3c2.slice/crio-e4bba444d50ce403c030a8d45ea1fe65c6616ec71bbdd3cda63a75c2b00d2267 WatchSource:0}: Error finding container e4bba444d50ce403c030a8d45ea1fe65c6616ec71bbdd3cda63a75c2b00d2267: Status 404 returned error can't find the container with id e4bba444d50ce403c030a8d45ea1fe65c6616ec71bbdd3cda63a75c2b00d2267 Jul 06 23:48:05.637176 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.637159 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:48:05.909399 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:05.909276 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:06.029722 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:06.029686 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:06.057518 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:06.057482 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" event={"ID":"2d5dc851-5a03-4587-8f36-f5408da0a3c2","Type":"ContainerStarted","Data":"e4bba444d50ce403c030a8d45ea1fe65c6616ec71bbdd3cda63a75c2b00d2267"} Jul 06 23:48:06.428033 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:06.427997 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:06.541058 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:06.541023 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:07.461308 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:07.461276 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:09.071691 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:09.071659 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" event={"ID":"2d5dc851-5a03-4587-8f36-f5408da0a3c2","Type":"ContainerStarted","Data":"03dba884f45730d0a37ed23f4c58f42fcb0e486488cc53ba0b162a4916fb4388"} Jul 06 23:48:09.088229 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:09.088163 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5646546ff9-ctv9q" podStartSLOduration=1.280332073 podStartE2EDuration="4.088146298s" totalImagesPullingTime="2.807814225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:09.086679336 +0000 UTC m=+317.978868013" watchObservedRunningTime="2026-07-06 23:48:09.088146298 +0000 UTC m=+317.980334976" Jul 06 23:48:12.880699 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:12.880664 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:12.975493 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:12.975461 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:13.447161 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.447129 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f"] Jul 06 23:48:13.450398 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.450382 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.453077 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.453050 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 23:48:13.453902 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.453885 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 23:48:13.454019 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.453913 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 23:48:13.454019 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.453972 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rhvnz\"" Jul 06 23:48:13.463028 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.463006 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f"] Jul 06 23:48:13.515470 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.515436 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9e739d36-6ad3-4689-a779-68f7f566c40c-manager-config\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.515470 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.515482 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzx65\" (UniqueName: \"kubernetes.io/projected/9e739d36-6ad3-4689-a779-68f7f566c40c-kube-api-access-vzx65\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.515758 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.515523 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e739d36-6ad3-4689-a779-68f7f566c40c-metrics-cert\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.515758 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.515553 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e739d36-6ad3-4689-a779-68f7f566c40c-cert\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.616305 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.616262 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e739d36-6ad3-4689-a779-68f7f566c40c-metrics-cert\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.616305 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.616310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e739d36-6ad3-4689-a779-68f7f566c40c-cert\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.616544 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.616347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9e739d36-6ad3-4689-a779-68f7f566c40c-manager-config\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.616544 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.616375 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzx65\" (UniqueName: \"kubernetes.io/projected/9e739d36-6ad3-4689-a779-68f7f566c40c-kube-api-access-vzx65\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.617139 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.617068 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9e739d36-6ad3-4689-a779-68f7f566c40c-manager-config\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.619146 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.619118 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e739d36-6ad3-4689-a779-68f7f566c40c-cert\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.619146 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.619143 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e739d36-6ad3-4689-a779-68f7f566c40c-metrics-cert\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.629365 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.629335 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzx65\" (UniqueName: \"kubernetes.io/projected/9e739d36-6ad3-4689-a779-68f7f566c40c-kube-api-access-vzx65\") pod \"lws-controller-manager-58544fcc46-ggx4f\" (UID: \"9e739d36-6ad3-4689-a779-68f7f566c40c\") " pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.759691 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.759657 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:13.853191 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.853149 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:13.923203 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:13.923146 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f"] Jul 06 23:48:13.925931 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:48:13.925890 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e739d36_6ad3_4689_a779_68f7f566c40c.slice/crio-f8f6754310d4d774b85c666d4db6973d537722a310caa79fbb770e3489f09f71 WatchSource:0}: Error finding container f8f6754310d4d774b85c666d4db6973d537722a310caa79fbb770e3489f09f71: Status 404 returned error can't find the container with id f8f6754310d4d774b85c666d4db6973d537722a310caa79fbb770e3489f09f71 Jul 06 23:48:14.050803 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:14.050708 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:14.094417 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:14.094385 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" event={"ID":"9e739d36-6ad3-4689-a779-68f7f566c40c","Type":"ContainerStarted","Data":"f8f6754310d4d774b85c666d4db6973d537722a310caa79fbb770e3489f09f71"} Jul 06 23:48:14.202686 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:14.202653 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:16.105759 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:16.105724 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" event={"ID":"9e739d36-6ad3-4689-a779-68f7f566c40c","Type":"ContainerStarted","Data":"1cdab7f13837cff742aaaf1b046ce6d732ac83dc509f8abda9b30076cf06a41e"} Jul 06 23:48:16.106296 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:16.105834 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:17.426040 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:17.426001 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:17.542832 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:17.542801 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:17.643228 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:17.643190 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:17.744790 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:17.744752 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:18.749321 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:18.749291 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:18.913552 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:18.913522 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:19.031711 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:19.031614 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:27.113106 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:27.113071 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" Jul 06 23:48:27.132984 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:27.132929 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58544fcc46-ggx4f" podStartSLOduration=12.491688655 podStartE2EDuration="14.132911575s" totalImagesPullingTime="1.64122292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:16.13993845 +0000 UTC m=+325.032127128" watchObservedRunningTime="2026-07-06 23:48:27.132911575 +0000 UTC m=+336.025100252" Jul 06 23:48:44.328028 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:44.327952 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:44.428782 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:44.428740 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:48:46.273820 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:48:46.273783 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:06.124892 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:06.124857 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:06.226122 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:06.226084 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:06.325873 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:06.325840 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:07.071241 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:07.071208 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:07.263149 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:07.263112 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:34.809960 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:34.809914 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:38.679619 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:38.679589 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:40.047122 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:40.047094 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:40.418084 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:40.418003 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:40.899247 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:40.899216 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:41.000062 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:41.000031 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:41.453333 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:41.453297 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:41.548721 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:41.548688 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:43.029863 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:43.029829 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:43.122456 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:43.122424 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:44.628721 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:44.628688 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx"] Jul 06 23:49:44.632057 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:44.632041 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx" Jul 06 23:49:44.635014 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:44.634994 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 23:49:44.636220 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:44.636203 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 23:49:44.636277 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:44.636224 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-8gkjt\"" Jul 06 23:49:44.636315 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:44.636281 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 23:49:44.643243 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:44.643216 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx"] Jul 06 23:49:44.802976 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:44.802939 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tr8c\" (UniqueName: \"kubernetes.io/projected/af9a8e1b-8f6e-4e3f-a27c-95089050849c-kube-api-access-4tr8c\") pod \"dns-operator-controller-manager-864dd78c4c-rp7cx\" (UID: \"af9a8e1b-8f6e-4e3f-a27c-95089050849c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx" Jul 06 23:49:44.904467 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:44.904393 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tr8c\" (UniqueName: \"kubernetes.io/projected/af9a8e1b-8f6e-4e3f-a27c-95089050849c-kube-api-access-4tr8c\") pod \"dns-operator-controller-manager-864dd78c4c-rp7cx\" (UID: \"af9a8e1b-8f6e-4e3f-a27c-95089050849c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx" Jul 06 23:49:44.915422 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:44.915394 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tr8c\" (UniqueName: \"kubernetes.io/projected/af9a8e1b-8f6e-4e3f-a27c-95089050849c-kube-api-access-4tr8c\") pod \"dns-operator-controller-manager-864dd78c4c-rp7cx\" (UID: \"af9a8e1b-8f6e-4e3f-a27c-95089050849c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx" Jul 06 23:49:44.942664 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:44.942642 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx" Jul 06 23:49:45.077526 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:45.077503 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx"] Jul 06 23:49:45.080298 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:49:45.080276 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9a8e1b_8f6e_4e3f_a27c_95089050849c.slice/crio-bc43a316d1881ed1927a0d0dc390a5bc3ad5d8ea3cc18c1f82a777b8d7347b70 WatchSource:0}: Error finding container bc43a316d1881ed1927a0d0dc390a5bc3ad5d8ea3cc18c1f82a777b8d7347b70: Status 404 returned error can't find the container with id bc43a316d1881ed1927a0d0dc390a5bc3ad5d8ea3cc18c1f82a777b8d7347b70 Jul 06 23:49:45.230050 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:45.230014 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:45.495019 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:45.494939 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx" event={"ID":"af9a8e1b-8f6e-4e3f-a27c-95089050849c","Type":"ContainerStarted","Data":"bc43a316d1881ed1927a0d0dc390a5bc3ad5d8ea3cc18c1f82a777b8d7347b70"} Jul 06 23:49:46.623032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:46.622999 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4"] Jul 06 23:49:46.626609 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:46.626565 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:49:46.628963 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:46.628940 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-cb456\"" Jul 06 23:49:46.637710 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:46.637655 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4"] Jul 06 23:49:46.721766 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:46.721735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrt72\" (UniqueName: \"kubernetes.io/projected/d8f5a432-ff49-45e8-89fb-5e459f9b67b5-kube-api-access-hrt72\") pod \"limitador-operator-controller-manager-859d7b78bd-8t5b4\" (UID: \"d8f5a432-ff49-45e8-89fb-5e459f9b67b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:49:46.823076 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:46.823040 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrt72\" (UniqueName: \"kubernetes.io/projected/d8f5a432-ff49-45e8-89fb-5e459f9b67b5-kube-api-access-hrt72\") pod \"limitador-operator-controller-manager-859d7b78bd-8t5b4\" (UID: \"d8f5a432-ff49-45e8-89fb-5e459f9b67b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:49:46.835477 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:46.835440 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrt72\" (UniqueName: \"kubernetes.io/projected/d8f5a432-ff49-45e8-89fb-5e459f9b67b5-kube-api-access-hrt72\") pod \"limitador-operator-controller-manager-859d7b78bd-8t5b4\" (UID: \"d8f5a432-ff49-45e8-89fb-5e459f9b67b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:49:46.940561 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:46.940472 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:49:47.090830 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:47.090807 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:47.167853 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:47.167831 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4"] Jul 06 23:49:47.169083 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:49:47.169051 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f5a432_ff49_45e8_89fb_5e459f9b67b5.slice/crio-a8fc91c4576656a2bf302f7b99802e3aeaa16159c59097b47e9a8a70437c3c5e WatchSource:0}: Error finding container a8fc91c4576656a2bf302f7b99802e3aeaa16159c59097b47e9a8a70437c3c5e: Status 404 returned error can't find the container with id a8fc91c4576656a2bf302f7b99802e3aeaa16159c59097b47e9a8a70437c3c5e Jul 06 23:49:47.504979 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:47.504944 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" event={"ID":"d8f5a432-ff49-45e8-89fb-5e459f9b67b5","Type":"ContainerStarted","Data":"a8fc91c4576656a2bf302f7b99802e3aeaa16159c59097b47e9a8a70437c3c5e"} Jul 06 23:49:47.506421 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:47.506395 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx" event={"ID":"af9a8e1b-8f6e-4e3f-a27c-95089050849c","Type":"ContainerStarted","Data":"e2c4c1c8f10ccf644bc03bfa7762790fdc130b801e95b206b41a7546290c59fa"} Jul 06 23:49:47.506519 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:47.506501 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx" Jul 06 23:49:47.531634 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:47.531586 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx" podStartSLOduration=1.528170818 podStartE2EDuration="3.531557188s" totalImagesPullingTime="2.00338637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:47.529357716 +0000 UTC m=+416.421546395" watchObservedRunningTime="2026-07-06 23:49:47.531557188 +0000 UTC m=+416.423745866" Jul 06 23:49:48.816280 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:48.816243 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:48.913118 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:48.913082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:49.272331 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.272301 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t"] Jul 06 23:49:49.275701 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.275680 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:49.278565 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.278546 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 23:49:49.278882 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.278859 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 23:49:49.279441 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.279428 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v84bn\"" Jul 06 23:49:49.293545 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.293508 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:49.305268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.305248 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t"] Jul 06 23:49:49.345456 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.345382 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/371d6297-c9a9-4518-8a3a-27e585b117ba-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8jv5t\" (UID: \"371d6297-c9a9-4518-8a3a-27e585b117ba\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:49.345456 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.345430 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/371d6297-c9a9-4518-8a3a-27e585b117ba-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8jv5t\" (UID: \"371d6297-c9a9-4518-8a3a-27e585b117ba\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:49.345637 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.345489 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4x4p\" (UniqueName: \"kubernetes.io/projected/371d6297-c9a9-4518-8a3a-27e585b117ba-kube-api-access-x4x4p\") pod \"kuadrant-console-plugin-66b5774985-8jv5t\" (UID: \"371d6297-c9a9-4518-8a3a-27e585b117ba\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:49.446312 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.446284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/371d6297-c9a9-4518-8a3a-27e585b117ba-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8jv5t\" (UID: \"371d6297-c9a9-4518-8a3a-27e585b117ba\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:49.446460 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.446323 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/371d6297-c9a9-4518-8a3a-27e585b117ba-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8jv5t\" (UID: \"371d6297-c9a9-4518-8a3a-27e585b117ba\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:49.446460 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.446342 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4x4p\" (UniqueName: \"kubernetes.io/projected/371d6297-c9a9-4518-8a3a-27e585b117ba-kube-api-access-x4x4p\") pod \"kuadrant-console-plugin-66b5774985-8jv5t\" (UID: \"371d6297-c9a9-4518-8a3a-27e585b117ba\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:49.446460 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:49:49.446442 2560 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 06 23:49:49.446557 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:49:49.446505 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/371d6297-c9a9-4518-8a3a-27e585b117ba-plugin-serving-cert podName:371d6297-c9a9-4518-8a3a-27e585b117ba nodeName:}" failed. No retries permitted until 2026-07-06 23:49:49.94648921 +0000 UTC m=+418.838677866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/371d6297-c9a9-4518-8a3a-27e585b117ba-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-8jv5t" (UID: "371d6297-c9a9-4518-8a3a-27e585b117ba") : secret "plugin-serving-cert" not found Jul 06 23:49:49.447194 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.447177 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/371d6297-c9a9-4518-8a3a-27e585b117ba-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8jv5t\" (UID: \"371d6297-c9a9-4518-8a3a-27e585b117ba\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:49.455318 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.455296 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4x4p\" (UniqueName: \"kubernetes.io/projected/371d6297-c9a9-4518-8a3a-27e585b117ba-kube-api-access-x4x4p\") pod \"kuadrant-console-plugin-66b5774985-8jv5t\" (UID: \"371d6297-c9a9-4518-8a3a-27e585b117ba\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:49.515974 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.515947 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" event={"ID":"d8f5a432-ff49-45e8-89fb-5e459f9b67b5","Type":"ContainerStarted","Data":"629882b869aaab8e4ad7e919c3512b538e5dab48f2dbb73f33ac61254a1c694e"} Jul 06 23:49:49.516103 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.516069 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:49:49.534613 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.534550 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" podStartSLOduration=1.61573415 podStartE2EDuration="3.534537144s" totalImagesPullingTime="1.918802994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:49.532967491 +0000 UTC m=+418.425156193" watchObservedRunningTime="2026-07-06 23:49:49.534537144 +0000 UTC m=+418.426725823" Jul 06 23:49:49.594521 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.594494 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:49.949433 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.949399 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/371d6297-c9a9-4518-8a3a-27e585b117ba-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8jv5t\" (UID: \"371d6297-c9a9-4518-8a3a-27e585b117ba\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:49.951895 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:49.951874 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/371d6297-c9a9-4518-8a3a-27e585b117ba-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8jv5t\" (UID: \"371d6297-c9a9-4518-8a3a-27e585b117ba\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:50.184719 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:50.184691 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" Jul 06 23:49:50.316306 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:50.316281 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t"] Jul 06 23:49:50.317789 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:49:50.317759 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod371d6297_c9a9_4518_8a3a_27e585b117ba.slice/crio-f189abc28e8cc736a94e8af332156675165aceeb3397587a720ac0b1e82fe539 WatchSource:0}: Error finding container f189abc28e8cc736a94e8af332156675165aceeb3397587a720ac0b1e82fe539: Status 404 returned error can't find the container with id f189abc28e8cc736a94e8af332156675165aceeb3397587a720ac0b1e82fe539 Jul 06 23:49:50.521470 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:50.521382 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" event={"ID":"371d6297-c9a9-4518-8a3a-27e585b117ba","Type":"ContainerStarted","Data":"f189abc28e8cc736a94e8af332156675165aceeb3397587a720ac0b1e82fe539"} Jul 06 23:49:58.111946 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:58.111897 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:58.213378 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:58.213330 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:58.339758 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:58.339715 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:49:58.514047 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:58.514017 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rp7cx" Jul 06 23:49:59.707801 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.707769 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4"] Jul 06 23:49:59.708253 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.708014 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" podUID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" containerName="manager" containerID="cri-o://629882b869aaab8e4ad7e919c3512b538e5dab48f2dbb73f33ac61254a1c694e" gracePeriod=2 Jul 06 23:49:59.710008 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.709988 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:49:59.730763 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.730720 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4"] Jul 06 23:49:59.738925 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:49:59.738886 2560 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d8f5a432-ff49-45e8-89fb-5e459f9b67b5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-8t5b4\": pods \"limitador-operator-controller-manager-859d7b78bd-8t5b4\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:49:59.743799 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:49:59.743762 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t5b4\" is forbidden: User \"system:node:ip-10-0-135-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-66.ec2.internal' and this object" podUID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:49:59.751026 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.750983 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt"] Jul 06 23:49:59.751544 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.751508 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" containerName="manager" Jul 06 23:49:59.751544 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.751538 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" containerName="manager" Jul 06 23:49:59.751766 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.751653 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" containerName="manager" Jul 06 23:49:59.756340 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.756320 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt" Jul 06 23:49:59.767146 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.767085 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt"] Jul 06 23:49:59.783085 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:49:59.783046 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t5b4\" is forbidden: User \"system:node:ip-10-0-135-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-66.ec2.internal' and this object" podUID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:49:59.842636 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.842605 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8gd5\" (UniqueName: \"kubernetes.io/projected/0df8224e-1240-40a0-837f-4d5ad17ce493-kube-api-access-g8gd5\") pod \"limitador-operator-controller-manager-859d7b78bd-tkprt\" (UID: \"0df8224e-1240-40a0-837f-4d5ad17ce493\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt" Jul 06 23:49:59.943144 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.943111 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8gd5\" (UniqueName: \"kubernetes.io/projected/0df8224e-1240-40a0-837f-4d5ad17ce493-kube-api-access-g8gd5\") pod \"limitador-operator-controller-manager-859d7b78bd-tkprt\" (UID: \"0df8224e-1240-40a0-837f-4d5ad17ce493\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt" Jul 06 23:49:59.956048 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:49:59.956017 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8gd5\" (UniqueName: \"kubernetes.io/projected/0df8224e-1240-40a0-837f-4d5ad17ce493-kube-api-access-g8gd5\") pod \"limitador-operator-controller-manager-859d7b78bd-tkprt\" (UID: \"0df8224e-1240-40a0-837f-4d5ad17ce493\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt" Jul 06 23:50:00.096417 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:00.096330 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt" Jul 06 23:50:00.587448 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:00.587414 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:00.690819 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:00.690655 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:01.562016 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:50:01.561973 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t5b4\" is forbidden: User \"system:node:ip-10-0-135-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-66.ec2.internal' and this object" podUID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:50:05.347094 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:05.347053 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:05.463724 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:05.463694 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:09.047089 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:09.047054 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:09.166228 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:09.166194 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:11.627719 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:11.627675 2560 generic.go:356] "Generic (PLEG): container finished" podID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" containerID="629882b869aaab8e4ad7e919c3512b538e5dab48f2dbb73f33ac61254a1c694e" exitCode=0 Jul 06 23:50:11.753655 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:11.753624 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt"] Jul 06 23:50:11.771407 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:11.771387 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:50:11.773314 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:50:11.773291 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t5b4\" is forbidden: User \"system:node:ip-10-0-135-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-66.ec2.internal' and this object" podUID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:50:11.853996 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:11.853966 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrt72\" (UniqueName: \"kubernetes.io/projected/d8f5a432-ff49-45e8-89fb-5e459f9b67b5-kube-api-access-hrt72\") pod \"d8f5a432-ff49-45e8-89fb-5e459f9b67b5\" (UID: \"d8f5a432-ff49-45e8-89fb-5e459f9b67b5\") " Jul 06 23:50:11.856037 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:11.856015 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8f5a432-ff49-45e8-89fb-5e459f9b67b5-kube-api-access-hrt72" (OuterVolumeSpecName: "kube-api-access-hrt72") pod "d8f5a432-ff49-45e8-89fb-5e459f9b67b5" (UID: "d8f5a432-ff49-45e8-89fb-5e459f9b67b5"). InnerVolumeSpecName "kube-api-access-hrt72". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:50:11.910051 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:50:11.910002 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df8224e_1240_40a0_837f_4d5ad17ce493.slice/crio-2611c440cbe18a739aa6ac4a816ed6d95f29a10b1577bd2cddbc49b1669ab9c7 WatchSource:0}: Error finding container 2611c440cbe18a739aa6ac4a816ed6d95f29a10b1577bd2cddbc49b1669ab9c7: Status 404 returned error can't find the container with id 2611c440cbe18a739aa6ac4a816ed6d95f29a10b1577bd2cddbc49b1669ab9c7 Jul 06 23:50:11.954947 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:11.954928 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrt72\" (UniqueName: \"kubernetes.io/projected/d8f5a432-ff49-45e8-89fb-5e459f9b67b5-kube-api-access-hrt72\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:50:12.633642 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:12.633612 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt" event={"ID":"0df8224e-1240-40a0-837f-4d5ad17ce493","Type":"ContainerStarted","Data":"bc57aa6acfb40b7b8cbb3d232e5b756ada1f2a6d24055c30cd413997ed2abdc4"} Jul 06 23:50:12.634013 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:12.633671 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt" event={"ID":"0df8224e-1240-40a0-837f-4d5ad17ce493","Type":"ContainerStarted","Data":"2611c440cbe18a739aa6ac4a816ed6d95f29a10b1577bd2cddbc49b1669ab9c7"} Jul 06 23:50:12.634013 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:12.633698 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt" Jul 06 23:50:12.635245 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:12.635219 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" event={"ID":"371d6297-c9a9-4518-8a3a-27e585b117ba","Type":"ContainerStarted","Data":"ef108ef83eb906b190b106c26a32e7367de0877908de2668230ed2542d24e920"} Jul 06 23:50:12.636471 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:12.636454 2560 scope.go:117] "RemoveContainer" containerID="629882b869aaab8e4ad7e919c3512b538e5dab48f2dbb73f33ac61254a1c694e" Jul 06 23:50:12.636591 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:12.636478 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:50:12.652027 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:12.651989 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt" podStartSLOduration=13.651975546 podStartE2EDuration="13.651975546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:12.651136211 +0000 UTC m=+441.543324890" watchObservedRunningTime="2026-07-06 23:50:12.651975546 +0000 UTC m=+441.544164223" Jul 06 23:50:12.652866 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:50:12.652846 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t5b4\" is forbidden: User \"system:node:ip-10-0-135-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-66.ec2.internal' and this object" podUID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:50:12.672899 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:50:12.672869 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t5b4\" is forbidden: User \"system:node:ip-10-0-135-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-66.ec2.internal' and this object" podUID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t5b4" Jul 06 23:50:12.673088 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:12.673058 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8jv5t" podStartSLOduration=2.027013082 podStartE2EDuration="23.673044885s" totalImagesPullingTime="21.646031803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:12.670848306 +0000 UTC m=+441.563036984" watchObservedRunningTime="2026-07-06 23:50:12.673044885 +0000 UTC m=+441.565233562" Jul 06 23:50:13.563073 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:13.563036 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8f5a432-ff49-45e8-89fb-5e459f9b67b5" path="/var/lib/kubelet/pods/d8f5a432-ff49-45e8-89fb-5e459f9b67b5/volumes" Jul 06 23:50:17.738733 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:17.738696 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:17.838344 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:17.838305 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:20.194089 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:20.194058 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:20.287422 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:20.287396 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:20.388125 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:20.388082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:23.644213 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:23.644178 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tkprt" Jul 06 23:50:24.988973 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:24.988942 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:25.083605 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:25.083553 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:28.466720 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:28.466681 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:28.567909 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:28.567880 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:28.963507 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:28.963478 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:29.064925 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:29.064899 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:29.477178 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:29.477141 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:29.889432 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:29.889354 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:31.304209 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:31.304179 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:31.404025 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:31.403989 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:31.566127 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:31.566043 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:32.060638 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.060604 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-tdvkd"] Jul 06 23:50:32.068375 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.068349 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tdvkd" Jul 06 23:50:32.070904 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.070878 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rhqch\"" Jul 06 23:50:32.081912 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.081892 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tdvkd"] Jul 06 23:50:32.127849 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.127812 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzlpt\" (UniqueName: \"kubernetes.io/projected/849f71ea-282b-4de4-8c6d-c65f78f13a59-kube-api-access-rzlpt\") pod \"authorino-58cd955675-tdvkd\" (UID: \"849f71ea-282b-4de4-8c6d-c65f78f13a59\") " pod="kuadrant-system/authorino-58cd955675-tdvkd" Jul 06 23:50:32.200711 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.200688 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:32.229217 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.229182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzlpt\" (UniqueName: \"kubernetes.io/projected/849f71ea-282b-4de4-8c6d-c65f78f13a59-kube-api-access-rzlpt\") pod \"authorino-58cd955675-tdvkd\" (UID: \"849f71ea-282b-4de4-8c6d-c65f78f13a59\") " pod="kuadrant-system/authorino-58cd955675-tdvkd" Jul 06 23:50:32.250225 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.250198 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzlpt\" (UniqueName: \"kubernetes.io/projected/849f71ea-282b-4de4-8c6d-c65f78f13a59-kube-api-access-rzlpt\") pod \"authorino-58cd955675-tdvkd\" (UID: \"849f71ea-282b-4de4-8c6d-c65f78f13a59\") " pod="kuadrant-system/authorino-58cd955675-tdvkd" Jul 06 23:50:32.334033 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.333953 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:32.377307 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.377286 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tdvkd" Jul 06 23:50:32.522712 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.522538 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tdvkd"] Jul 06 23:50:32.524899 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:50:32.524874 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod849f71ea_282b_4de4_8c6d_c65f78f13a59.slice/crio-10eb60998a6c7b052321b45b0634e24725b9eebc8ed246450e2182c3a87d0fa2 WatchSource:0}: Error finding container 10eb60998a6c7b052321b45b0634e24725b9eebc8ed246450e2182c3a87d0fa2: Status 404 returned error can't find the container with id 10eb60998a6c7b052321b45b0634e24725b9eebc8ed246450e2182c3a87d0fa2 Jul 06 23:50:32.728232 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:32.728200 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tdvkd" event={"ID":"849f71ea-282b-4de4-8c6d-c65f78f13a59","Type":"ContainerStarted","Data":"10eb60998a6c7b052321b45b0634e24725b9eebc8ed246450e2182c3a87d0fa2"} Jul 06 23:50:36.754470 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:36.754434 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tdvkd" event={"ID":"849f71ea-282b-4de4-8c6d-c65f78f13a59","Type":"ContainerStarted","Data":"9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96"} Jul 06 23:50:36.771939 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:36.771870 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-tdvkd" podStartSLOduration=1.437295403 podStartE2EDuration="4.77184901s" totalImagesPullingTime="3.334553607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:36.770788754 +0000 UTC m=+465.662977431" watchObservedRunningTime="2026-07-06 23:50:36.77184901 +0000 UTC m=+465.664037692" Jul 06 23:50:36.806948 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:36.806912 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:37.192658 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:37.192582 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tdvkd"] Jul 06 23:50:38.764102 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:38.764057 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-tdvkd" podUID="849f71ea-282b-4de4-8c6d-c65f78f13a59" containerName="authorino" containerID="cri-o://9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96" gracePeriod=30 Jul 06 23:50:39.010557 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.010532 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tdvkd" Jul 06 23:50:39.094806 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.094778 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzlpt\" (UniqueName: \"kubernetes.io/projected/849f71ea-282b-4de4-8c6d-c65f78f13a59-kube-api-access-rzlpt\") pod \"849f71ea-282b-4de4-8c6d-c65f78f13a59\" (UID: \"849f71ea-282b-4de4-8c6d-c65f78f13a59\") " Jul 06 23:50:39.096914 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.096887 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/849f71ea-282b-4de4-8c6d-c65f78f13a59-kube-api-access-rzlpt" (OuterVolumeSpecName: "kube-api-access-rzlpt") pod "849f71ea-282b-4de4-8c6d-c65f78f13a59" (UID: "849f71ea-282b-4de4-8c6d-c65f78f13a59"). InnerVolumeSpecName "kube-api-access-rzlpt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:50:39.148232 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.148202 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:39.195515 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.195456 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzlpt\" (UniqueName: \"kubernetes.io/projected/849f71ea-282b-4de4-8c6d-c65f78f13a59-kube-api-access-rzlpt\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 06 23:50:39.263485 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.263455 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:39.405290 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.405260 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:39.509900 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.509873 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:39.769746 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.769666 2560 generic.go:356] "Generic (PLEG): container finished" podID="849f71ea-282b-4de4-8c6d-c65f78f13a59" containerID="9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96" exitCode=0 Jul 06 23:50:39.769746 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.769708 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tdvkd" Jul 06 23:50:39.769746 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.769738 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tdvkd" event={"ID":"849f71ea-282b-4de4-8c6d-c65f78f13a59","Type":"ContainerDied","Data":"9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96"} Jul 06 23:50:39.770240 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.769770 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tdvkd" event={"ID":"849f71ea-282b-4de4-8c6d-c65f78f13a59","Type":"ContainerDied","Data":"10eb60998a6c7b052321b45b0634e24725b9eebc8ed246450e2182c3a87d0fa2"} Jul 06 23:50:39.770240 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.769792 2560 scope.go:117] "RemoveContainer" containerID="9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96" Jul 06 23:50:39.778130 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.778114 2560 scope.go:117] "RemoveContainer" containerID="9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96" Jul 06 23:50:39.778388 ip-10-0-135-66 kubenswrapper[2560]: E0706 23:50:39.778371 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96\": container with ID starting with 9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96 not found: ID does not exist" containerID="9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96" Jul 06 23:50:39.778449 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.778394 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96"} err="failed to get container status \"9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96\": rpc error: code = NotFound desc = could not find container \"9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96\": container with ID starting with 9d52b283d420951d48e7f63b15b779904e1b8895e07d86b91a4310a03b4acd96 not found: ID does not exist" Jul 06 23:50:39.788081 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.788059 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tdvkd"] Jul 06 23:50:39.793933 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:39.793914 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-tdvkd"] Jul 06 23:50:40.943216 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:40.943183 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:41.044521 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:41.044488 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:41.563027 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:41.562993 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="849f71ea-282b-4de4-8c6d-c65f78f13a59" path="/var/lib/kubelet/pods/849f71ea-282b-4de4-8c6d-c65f78f13a59/volumes" Jul 06 23:50:45.382217 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:45.382182 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:45.483991 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:45.483961 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:47.789359 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:47.789329 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:59.587256 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:59.587198 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:50:59.945315 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:50:59.945242 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:05.896273 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:05.896241 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:05.996725 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:05.996687 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:06.093193 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:06.093159 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:06.229119 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:06.229089 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:09.619114 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:09.619076 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:09.714243 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:09.714212 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:09.861550 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:09.861517 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:14.881493 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:14.881454 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:28.360953 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:28.360918 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:40.991726 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:40.991688 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:41.551678 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:41.551642 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:42.069154 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:42.069123 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:42.237815 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:42.237775 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:42.399180 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:42.399090 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:52.580165 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:52.580130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:51:52.706549 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:51:52.706515 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:00.952792 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:00.952753 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:14.551761 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:14.551726 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:15.807086 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:15.807058 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:22.331399 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.331360 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb"] Jul 06 23:52:22.331772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.331759 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="849f71ea-282b-4de4-8c6d-c65f78f13a59" containerName="authorino" Jul 06 23:52:22.331772 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.331774 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="849f71ea-282b-4de4-8c6d-c65f78f13a59" containerName="authorino" Jul 06 23:52:22.331844 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.331824 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="849f71ea-282b-4de4-8c6d-c65f78f13a59" containerName="authorino" Jul 06 23:52:22.334746 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.334731 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.341398 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.341375 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 23:52:22.341501 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.341375 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 23:52:22.341501 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.341423 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4n7l9\"" Jul 06 23:52:22.341768 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.341750 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 23:52:22.351315 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.351293 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb"] Jul 06 23:52:22.362056 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.362034 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.362170 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.362081 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9lcd\" (UniqueName: \"kubernetes.io/projected/03c5e60a-6af7-4a21-8631-06c304998c3c-kube-api-access-c9lcd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.362232 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.362178 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03c5e60a-6af7-4a21-8631-06c304998c3c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.362232 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.362212 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.362343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.362239 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.362343 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.362271 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.411443 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.411418 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:22.463490 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.463467 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.463610 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.463494 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.463610 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.463513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.463610 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.463544 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.463610 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.463594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9lcd\" (UniqueName: \"kubernetes.io/projected/03c5e60a-6af7-4a21-8631-06c304998c3c-kube-api-access-c9lcd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.463796 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.463639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03c5e60a-6af7-4a21-8631-06c304998c3c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.463918 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.463897 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.463998 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.463983 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.464047 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.463973 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.465923 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.465902 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03c5e60a-6af7-4a21-8631-06c304998c3c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.466020 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.466006 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03c5e60a-6af7-4a21-8631-06c304998c3c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.473909 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.473884 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9lcd\" (UniqueName: \"kubernetes.io/projected/03c5e60a-6af7-4a21-8631-06c304998c3c-kube-api-access-c9lcd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kvhfb\" (UID: \"03c5e60a-6af7-4a21-8631-06c304998c3c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.644422 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.644350 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:22.800287 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:22.800256 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb"] Jul 06 23:52:22.803009 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:52:22.802979 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c5e60a_6af7_4a21_8631_06c304998c3c.slice/crio-3ded957d04613d1e26a1a852287e7892309663254d8ee835f4f8c6d12041f407 WatchSource:0}: Error finding container 3ded957d04613d1e26a1a852287e7892309663254d8ee835f4f8c6d12041f407: Status 404 returned error can't find the container with id 3ded957d04613d1e26a1a852287e7892309663254d8ee835f4f8c6d12041f407 Jul 06 23:52:23.219485 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:23.219446 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" event={"ID":"03c5e60a-6af7-4a21-8631-06c304998c3c","Type":"ContainerStarted","Data":"3ded957d04613d1e26a1a852287e7892309663254d8ee835f4f8c6d12041f407"} Jul 06 23:52:24.226284 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:24.226249 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" event={"ID":"03c5e60a-6af7-4a21-8631-06c304998c3c","Type":"ContainerStarted","Data":"448968be2c7ac8a3b090f5c45b89b835c19abcd4cad924f7120992dd52669db1"} Jul 06 23:52:32.645032 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:32.645001 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:32.645425 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:32.645076 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:32.657672 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:32.657649 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:32.713153 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:32.713106 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" podStartSLOduration=9.685348376 podStartE2EDuration="10.713091063s" totalImagesPullingTime="1.027742687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:52:24.254526959 +0000 UTC m=+573.146715637" watchObservedRunningTime="2026-07-06 23:52:32.713091063 +0000 UTC m=+581.605279741" Jul 06 23:52:32.793145 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:32.793115 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:32.978804 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:32.978753 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:33.276988 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:33.276913 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kvhfb" Jul 06 23:52:37.703247 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.703220 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk"] Jul 06 23:52:37.706638 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.706622 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.715825 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.715805 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 06 23:52:37.747487 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.747461 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk"] Jul 06 23:52:37.794876 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.794850 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghmgh\" (UniqueName: \"kubernetes.io/projected/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-kube-api-access-ghmgh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.794984 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.794895 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.794984 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.794913 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.794984 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.794955 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.795092 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.795000 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.795092 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.795042 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.802794 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.802724 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:37.896268 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.896234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.896420 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.896293 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghmgh\" (UniqueName: \"kubernetes.io/projected/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-kube-api-access-ghmgh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.896420 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.896338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.896420 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.896354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.896420 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.896375 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.896420 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.896414 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.896764 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.896728 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.896764 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.896743 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.896911 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.896812 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.898799 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.898779 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.898953 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.898938 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:37.909752 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:37.909732 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghmgh\" (UniqueName: \"kubernetes.io/projected/1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7-kube-api-access-ghmgh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk\" (UID: \"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:38.016200 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:38.016145 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:38.185075 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:52:38.185039 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0ee9fd_5246_4e3d_a6bb_2ee3f457d9b7.slice/crio-f3321f366d3aa41b4b45450ee48575e3e2877ea9839c343deb7d700775d5ee5f WatchSource:0}: Error finding container f3321f366d3aa41b4b45450ee48575e3e2877ea9839c343deb7d700775d5ee5f: Status 404 returned error can't find the container with id f3321f366d3aa41b4b45450ee48575e3e2877ea9839c343deb7d700775d5ee5f Jul 06 23:52:38.187324 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:38.187303 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk"] Jul 06 23:52:38.289541 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:38.289477 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" event={"ID":"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7","Type":"ContainerStarted","Data":"f3321f366d3aa41b4b45450ee48575e3e2877ea9839c343deb7d700775d5ee5f"} Jul 06 23:52:39.295795 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:39.295764 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" event={"ID":"1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7","Type":"ContainerStarted","Data":"94b044e5127ad597258caab60b3a229037bc77e71bb45f8a005ec8a6ba5544b8"} Jul 06 23:52:45.440546 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.440493 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" podStartSLOduration=8.267658192 podStartE2EDuration="8.440476295s" totalImagesPullingTime="172.818103ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:52:39.351191192 +0000 UTC m=+588.243379871" watchObservedRunningTime="2026-07-06 23:52:45.440476295 +0000 UTC m=+594.332665000" Jul 06 23:52:45.441698 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.441679 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw"] Jul 06 23:52:45.444604 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.444587 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.458956 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.458939 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 23:52:45.515483 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.515460 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw"] Jul 06 23:52:45.564076 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.564057 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.564165 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.564111 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpzfx\" (UniqueName: \"kubernetes.io/projected/fa037dd6-0e65-43d3-9e1c-fee0548b924b-kube-api-access-jpzfx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.564165 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.564130 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.564165 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.564147 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.564276 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.564189 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.564276 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.564211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa037dd6-0e65-43d3-9e1c-fee0548b924b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.613978 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.613956 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:45.665541 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.665514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpzfx\" (UniqueName: \"kubernetes.io/projected/fa037dd6-0e65-43d3-9e1c-fee0548b924b-kube-api-access-jpzfx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.665651 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.665547 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.665651 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.665591 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.665651 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.665636 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.665778 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.665674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa037dd6-0e65-43d3-9e1c-fee0548b924b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.665778 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.665736 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.666025 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.666002 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.666025 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.666016 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.666149 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.666087 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.668139 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.668120 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa037dd6-0e65-43d3-9e1c-fee0548b924b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.668328 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.668309 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa037dd6-0e65-43d3-9e1c-fee0548b924b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.681047 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.681021 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpzfx\" (UniqueName: \"kubernetes.io/projected/fa037dd6-0e65-43d3-9e1c-fee0548b924b-kube-api-access-jpzfx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9hwmw\" (UID: \"fa037dd6-0e65-43d3-9e1c-fee0548b924b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.755085 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.755064 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:45.904481 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:45.904461 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw"] Jul 06 23:52:45.907059 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:52:45.907030 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa037dd6_0e65_43d3_9e1c_fee0548b924b.slice/crio-35f3aeaf7a99bfd787a2b5410c8d14722fb1659d8e6a6b66b2da5e83f032a2f4 WatchSource:0}: Error finding container 35f3aeaf7a99bfd787a2b5410c8d14722fb1659d8e6a6b66b2da5e83f032a2f4: Status 404 returned error can't find the container with id 35f3aeaf7a99bfd787a2b5410c8d14722fb1659d8e6a6b66b2da5e83f032a2f4 Jul 06 23:52:46.330976 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:46.330887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" event={"ID":"fa037dd6-0e65-43d3-9e1c-fee0548b924b","Type":"ContainerStarted","Data":"00f58ac0546001b3e777fdbbf5db64b239fbc583477a835851b4fc98623b2db2"} Jul 06 23:52:46.330976 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:46.330936 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" event={"ID":"fa037dd6-0e65-43d3-9e1c-fee0548b924b","Type":"ContainerStarted","Data":"35f3aeaf7a99bfd787a2b5410c8d14722fb1659d8e6a6b66b2da5e83f032a2f4"} Jul 06 23:52:46.363837 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:46.363790 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" podStartSLOduration=1.197194909 podStartE2EDuration="1.363774904s" totalImagesPullingTime="166.579995ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:52:46.363033944 +0000 UTC m=+595.255222623" watchObservedRunningTime="2026-07-06 23:52:46.363774904 +0000 UTC m=+595.255963583" Jul 06 23:52:48.017045 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:48.017009 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:48.017480 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:48.017065 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:48.029565 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:48.029543 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:48.350626 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:48.350522 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk" Jul 06 23:52:55.005414 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.005379 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq"] Jul 06 23:52:55.010009 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.009994 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.014954 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.014932 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 06 23:52:55.038874 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.038853 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq"] Jul 06 23:52:55.148026 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.147987 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:52:55.149191 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.149171 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.149269 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.149201 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6zff\" (UniqueName: \"kubernetes.io/projected/78eaafa9-e153-4a67-b209-a290b85c4d4a-kube-api-access-w6zff\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.149269 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.149221 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.149373 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.149281 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.149432 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.149374 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78eaafa9-e153-4a67-b209-a290b85c4d4a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.149432 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.149411 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.249852 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.249824 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78eaafa9-e153-4a67-b209-a290b85c4d4a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.249981 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.249860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.249981 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.249918 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.249981 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.249942 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6zff\" (UniqueName: \"kubernetes.io/projected/78eaafa9-e153-4a67-b209-a290b85c4d4a-kube-api-access-w6zff\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.249981 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.249970 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.250172 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.250069 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.250395 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.250365 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.250498 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.250408 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.250560 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.250544 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.252289 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.252272 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78eaafa9-e153-4a67-b209-a290b85c4d4a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.252439 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.252423 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78eaafa9-e153-4a67-b209-a290b85c4d4a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.275007 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.274950 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6zff\" (UniqueName: \"kubernetes.io/projected/78eaafa9-e153-4a67-b209-a290b85c4d4a-kube-api-access-w6zff\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq\" (UID: \"78eaafa9-e153-4a67-b209-a290b85c4d4a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.318307 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.318284 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:52:55.471503 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.471469 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq"] Jul 06 23:52:55.475088 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:52:55.475048 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78eaafa9_e153_4a67_b209_a290b85c4d4a.slice/crio-ad0705f1f767bbe0937ad2acb4864ca15c07ea3d6aeb63ea8be5e3cab8a51f92 WatchSource:0}: Error finding container ad0705f1f767bbe0937ad2acb4864ca15c07ea3d6aeb63ea8be5e3cab8a51f92: Status 404 returned error can't find the container with id ad0705f1f767bbe0937ad2acb4864ca15c07ea3d6aeb63ea8be5e3cab8a51f92 Jul 06 23:52:55.755562 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.755523 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:55.755710 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.755602 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:55.769905 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:55.769878 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:56.379428 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:56.379392 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" event={"ID":"78eaafa9-e153-4a67-b209-a290b85c4d4a","Type":"ContainerStarted","Data":"0f42c0552f97dedfb490b8378a4e60ff44508e7ebe1f6ea5f8e55f871d7913d5"} Jul 06 23:52:56.379874 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:56.379435 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" event={"ID":"78eaafa9-e153-4a67-b209-a290b85c4d4a","Type":"ContainerStarted","Data":"ad0705f1f767bbe0937ad2acb4864ca15c07ea3d6aeb63ea8be5e3cab8a51f92"} Jul 06 23:52:56.390496 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:56.390474 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9hwmw" Jul 06 23:52:56.425014 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:52:56.424971 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" podStartSLOduration=2.246234651 podStartE2EDuration="2.424956293s" totalImagesPullingTime="178.721642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:52:56.408845854 +0000 UTC m=+605.301034533" watchObservedRunningTime="2026-07-06 23:52:56.424956293 +0000 UTC m=+605.317145020" Jul 06 23:53:05.319348 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:53:05.319314 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:53:05.319832 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:53:05.319379 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:53:05.331720 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:53:05.331691 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:53:05.433157 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:53:05.433132 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq" Jul 06 23:54:01.563235 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.563204 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67c9bcb79d-bnxlt"] Jul 06 23:54:01.566612 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.566597 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67c9bcb79d-bnxlt" Jul 06 23:54:01.569735 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.569547 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rhqch\"" Jul 06 23:54:01.569906 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.569884 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 23:54:01.571200 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.571177 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67c9bcb79d-bnxlt"] Jul 06 23:54:01.611158 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.611136 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64v9x\" (UniqueName: \"kubernetes.io/projected/12527d13-c1a8-4a92-a9a2-8d31c6791614-kube-api-access-64v9x\") pod \"authorino-67c9bcb79d-bnxlt\" (UID: \"12527d13-c1a8-4a92-a9a2-8d31c6791614\") " pod="kuadrant-system/authorino-67c9bcb79d-bnxlt" Jul 06 23:54:01.611272 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.611232 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/12527d13-c1a8-4a92-a9a2-8d31c6791614-tls-cert\") pod \"authorino-67c9bcb79d-bnxlt\" (UID: \"12527d13-c1a8-4a92-a9a2-8d31c6791614\") " pod="kuadrant-system/authorino-67c9bcb79d-bnxlt" Jul 06 23:54:01.711907 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.711883 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/12527d13-c1a8-4a92-a9a2-8d31c6791614-tls-cert\") pod \"authorino-67c9bcb79d-bnxlt\" (UID: \"12527d13-c1a8-4a92-a9a2-8d31c6791614\") " pod="kuadrant-system/authorino-67c9bcb79d-bnxlt" Jul 06 23:54:01.712014 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.711941 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64v9x\" (UniqueName: \"kubernetes.io/projected/12527d13-c1a8-4a92-a9a2-8d31c6791614-kube-api-access-64v9x\") pod \"authorino-67c9bcb79d-bnxlt\" (UID: \"12527d13-c1a8-4a92-a9a2-8d31c6791614\") " pod="kuadrant-system/authorino-67c9bcb79d-bnxlt" Jul 06 23:54:01.714356 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.714337 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/12527d13-c1a8-4a92-a9a2-8d31c6791614-tls-cert\") pod \"authorino-67c9bcb79d-bnxlt\" (UID: \"12527d13-c1a8-4a92-a9a2-8d31c6791614\") " pod="kuadrant-system/authorino-67c9bcb79d-bnxlt" Jul 06 23:54:01.719730 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.719711 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64v9x\" (UniqueName: \"kubernetes.io/projected/12527d13-c1a8-4a92-a9a2-8d31c6791614-kube-api-access-64v9x\") pod \"authorino-67c9bcb79d-bnxlt\" (UID: \"12527d13-c1a8-4a92-a9a2-8d31c6791614\") " pod="kuadrant-system/authorino-67c9bcb79d-bnxlt" Jul 06 23:54:01.877454 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.877399 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67c9bcb79d-bnxlt" Jul 06 23:54:01.933031 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:01.933007 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:02.008600 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:02.008558 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67c9bcb79d-bnxlt"] Jul 06 23:54:02.010506 ip-10-0-135-66 kubenswrapper[2560]: W0706 23:54:02.010479 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12527d13_c1a8_4a92_a9a2_8d31c6791614.slice/crio-d07f9b55418ae6457f32ffe493780219997ac00afcd55c4b7570e3e346ae60fc WatchSource:0}: Error finding container d07f9b55418ae6457f32ffe493780219997ac00afcd55c4b7570e3e346ae60fc: Status 404 returned error can't find the container with id d07f9b55418ae6457f32ffe493780219997ac00afcd55c4b7570e3e346ae60fc Jul 06 23:54:02.010714 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:02.010693 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:02.011921 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:02.011902 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:54:02.678757 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:02.678670 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67c9bcb79d-bnxlt" event={"ID":"12527d13-c1a8-4a92-a9a2-8d31c6791614","Type":"ContainerStarted","Data":"1adca1da8db980f40becc784ebe7526abe4bff04816fa3aa72f1e3224fb48bcb"} Jul 06 23:54:02.678757 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:02.678707 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67c9bcb79d-bnxlt" event={"ID":"12527d13-c1a8-4a92-a9a2-8d31c6791614","Type":"ContainerStarted","Data":"d07f9b55418ae6457f32ffe493780219997ac00afcd55c4b7570e3e346ae60fc"} Jul 06 23:54:02.701992 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:02.701944 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-67c9bcb79d-bnxlt" podStartSLOduration=1.332292535 podStartE2EDuration="1.701927646s" totalImagesPullingTime="369.635111ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:02.70068105 +0000 UTC m=+671.592869731" watchObservedRunningTime="2026-07-06 23:54:02.701927646 +0000 UTC m=+671.594116325" Jul 06 23:54:03.063879 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:03.063843 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:54:03.172927 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:54:03.172894 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:13.646541 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:55:13.646420 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:15.162084 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:55:15.162042 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:54.091310 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:55:54.091267 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:54.182550 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:55:54.182505 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:54.304823 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:55:54.304783 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:55:55.293072 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:55:55.293043 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:00.333825 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:56:00.333793 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:00.435841 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:56:00.435799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:00.533776 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:56:00.533742 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:00.682486 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:56:00.682380 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:06.375543 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:56:06.375496 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:56:06.448018 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:56:06.447981 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:30.186400 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:57:30.186364 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:39.656454 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:57:39.656399 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:57:39.712241 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:57:39.712208 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 06 23:59:37.577331 ip-10-0-135-66 kubenswrapper[2560]: I0706 23:59:37.577295 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:03:11.291976 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:03:11.291944 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:03:12.830103 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:03:12.830045 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:04:27.334489 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:04:27.334446 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:06:40.973070 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:06:40.973037 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:06:41.054531 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:06:41.054496 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:06:46.162812 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:06:46.162721 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:06:46.264392 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:06:46.264358 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:06:47.213808 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:06:47.213773 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:06:55.495075 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:06:55.495043 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:01.755189 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:01.755154 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:03.770220 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:03.770188 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:03.868464 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:03.868431 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:04.222283 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:04.222257 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:09.958922 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:09.958890 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:10.059743 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:10.059711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:10.187726 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:10.187691 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:10.330978 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:10.330903 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:11.365771 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:11.365738 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:11.457742 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:11.457707 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:11.527383 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:11.527349 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:11.627282 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:11.627200 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:11.942059 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:11.941972 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:12.041367 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:12.041336 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:13.681634 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:13.681590 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:16.484321 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:16.484283 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:16.698920 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:16.698884 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:16.802588 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:16.802495 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:16.858353 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:16.858325 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:20.204714 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:20.204670 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:22.795830 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:22.795795 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:22.895427 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:22.895393 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:23.030680 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:23.030646 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:23.131050 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:23.130960 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:24.219741 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:24.219703 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:24.316221 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:24.316182 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:24.413526 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:24.413496 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:24.518976 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:24.518905 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:24.794013 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:24.793936 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:24.891259 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:24.891226 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:26.394288 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:26.394237 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:29.402849 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:29.402814 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:29.757111 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:29.757077 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:35.526036 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:35.526003 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:35.623995 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:35.623964 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:37.452931 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:37.452896 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:41.669129 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:41.669093 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:41.732729 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:41.732701 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:41.862559 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:41.862531 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:41.962737 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:41.962705 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:42.962416 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:42.962385 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:43.051334 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:43.051304 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:43.146315 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:43.146282 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:43.513810 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:43.513775 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:43.712489 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:43.712456 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:43.812153 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:43.812075 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:45.778995 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:45.778965 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:47.484230 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:47.484202 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:47.627305 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:47.627271 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:47.727054 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:47.727021 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:47.828672 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:47.828588 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:49.313907 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:49.313876 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:53.377360 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:53.377326 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:53.476999 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:53.476961 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:53.874551 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:53.874517 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:53.972688 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:53.972661 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:54.137506 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:54.137432 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:07:55.567062 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:07:55.567027 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:01.461622 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:01.461555 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:01.529144 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:01.529110 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:01.639880 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:01.639829 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:01.739401 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:01.739309 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:01.839108 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:01.839072 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:02.298451 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:02.298417 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:28.793284 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:28.793248 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:34.199510 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:34.199472 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:34.308177 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:34.308146 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:34.403355 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:34.403323 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:50.330018 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:50.329986 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:56.035985 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:56.035948 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:56.139697 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:56.139663 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:08:56.235931 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:08:56.235899 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:06.769031 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:06.768997 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:07.628746 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:07.628708 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:13.557237 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:13.557182 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:13.671635 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:13.671604 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:13.767541 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:13.767503 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:13.865611 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:13.865486 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:14.640871 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:14.640827 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:14.740593 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:14.740530 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:14.838244 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:14.838212 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:16.446402 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:16.446363 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:22.169311 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:22.169266 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:22.265596 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:22.265538 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:22.845502 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:22.845470 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:28.899113 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:28.899073 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:28.972434 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:28.972402 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:29.199724 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:29.199640 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:29.511621 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:29.511588 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:29.640829 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:29.640795 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:32.270237 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:32.270205 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:32.369354 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:32.369324 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:32.493051 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:32.493018 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:32.738469 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:32.738437 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:32.925357 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:32.925319 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:33.018232 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:33.018116 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:36.796489 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:36.796447 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:37.514286 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:37.514249 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:37.608842 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:37.608805 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:37.969985 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:37.969954 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:43.855226 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:43.855137 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:43.914072 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:43.914040 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:44.304166 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:44.304134 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:44.611170 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:44.611084 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:44.762370 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:44.762339 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:50.715824 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:50.715780 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:50.797822 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:50.797789 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:50.994788 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:50.994706 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:51.404990 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:51.404908 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:51.504337 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:51.504302 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:51.604282 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:51.604247 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:57.339520 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:57.339484 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:57.447263 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:57.447230 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:57.546510 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:57.546483 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:09:57.712729 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:09:57.712699 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:03.601313 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:03.601282 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:03.701813 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:03.701765 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:10.578234 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:10.578198 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:11.035242 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:11.035210 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:11.199846 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:11.199807 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:14.419673 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:14.419633 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:14.529954 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:14.529921 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:14.624142 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:14.624114 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:15.043921 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:15.043875 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:15.144242 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:15.144208 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:15.251667 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:15.251632 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:18.861391 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:18.861355 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:25.211945 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:25.211900 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:25.890128 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:25.890092 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:36.218635 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:36.218595 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:36.319305 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:36.319259 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:36.978767 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:36.978729 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:37.084922 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:37.084890 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:46.240757 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:46.240723 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:46.679019 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:46.678944 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:52.431489 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:52.431451 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:10:57.508131 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:10:57.508094 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:34.121762 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:34.121729 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:34.491155 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:34.491125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:40.159764 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:40.159732 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:54.443757 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:54.443723 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:54.503911 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:54.503878 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:54.754267 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:54.754225 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:55.018468 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:55.018384 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:56.182026 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:56.181991 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:56.281050 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:56.281015 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:56.380968 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:56.380934 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:57.201815 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:57.201779 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:57.296221 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:57.296180 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:57.400643 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:57.400619 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:11:58.929225 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:11:58.929183 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:04.902799 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:04.902764 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:05.002989 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:05.002955 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:05.658224 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:05.658183 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:07.378386 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:07.378353 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:11.374427 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:11.374390 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:11.483636 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:11.483604 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:11.588452 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:11.588422 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:22.150681 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:22.150639 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:22.552221 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:22.552189 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:43.566758 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:43.566681 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:49.205844 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:49.205807 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:49.272261 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:49.272231 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:49.579081 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:49.579054 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:49.914933 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:49.914855 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:50.143340 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:50.143302 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:53.239274 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:53.239240 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:53.339647 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:53.339614 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:53.444815 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:53.444754 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:53.539825 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:53.539742 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:53.801968 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:53.801892 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:53.901064 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:53.901029 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:57.771986 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:57.771952 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:58.140666 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:58.140585 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:58.460953 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:58.460923 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:12:58.981397 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:12:58.981364 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:01.241765 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:01.241731 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:01.326287 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:01.326252 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:01.431605 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:01.431556 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:01.593952 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:01.593872 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:01.695005 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:01.694969 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:01.794212 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:01.794180 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:01.917295 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:01.917214 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:01.982351 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:01.982322 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:05.761770 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:05.761738 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:14.139835 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.139798 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd"] Jul 07 00:13:14.143585 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.143549 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.147259 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.147230 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-02e2a5\"/\"openshift-service-ca.crt\"" Jul 07 00:13:14.147370 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.147261 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-02e2a5\"/\"kube-root-ca.crt\"" Jul 07 00:13:14.147370 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.147319 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-02e2a5\"/\"test-model-02e2a5-kserve-self-signed-certs\"" Jul 07 00:13:14.147370 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.147261 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-02e2a5\"/\"default-dockercfg-kwrmk\"" Jul 07 00:13:14.154514 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.154490 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd"] Jul 07 00:13:14.180004 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.179980 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:14.203117 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.203091 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/030205c0-753a-4818-b123-49fa19dc30f3-tls-certs\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.203204 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.203127 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-home\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.203204 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.203200 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-model-cache\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.203278 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.203239 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-tmp-dir\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.203278 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.203264 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-dshm\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.203357 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.203290 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7wps\" (UniqueName: \"kubernetes.io/projected/030205c0-753a-4818-b123-49fa19dc30f3-kube-api-access-g7wps\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.304665 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.304634 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-home\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.304807 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.304707 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-model-cache\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.304807 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.304729 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-tmp-dir\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.304807 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.304747 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-dshm\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.304807 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.304774 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7wps\" (UniqueName: \"kubernetes.io/projected/030205c0-753a-4818-b123-49fa19dc30f3-kube-api-access-g7wps\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.304807 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.304793 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/030205c0-753a-4818-b123-49fa19dc30f3-tls-certs\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.305114 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.305092 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-tmp-dir\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.305167 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.305103 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-home\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.305226 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.305208 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-model-cache\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.307115 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.307095 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-dshm\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.307367 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.307349 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/030205c0-753a-4818-b123-49fa19dc30f3-tls-certs\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.319623 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.314881 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7wps\" (UniqueName: \"kubernetes.io/projected/030205c0-753a-4818-b123-49fa19dc30f3-kube-api-access-g7wps\") pod \"test-model-02e2a5-kserve-58776b5b6c-fs9bd\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.453709 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.453676 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:14.584621 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:14.584596 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd"] Jul 07 00:13:14.587206 ip-10-0-135-66 kubenswrapper[2560]: W0707 00:13:14.587173 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod030205c0_753a_4818_b123_49fa19dc30f3.slice/crio-8f531d5dc2afe9dd137caf5f8fec165618627236fb423be60eaedfb98b3dcd67 WatchSource:0}: Error finding container 8f531d5dc2afe9dd137caf5f8fec165618627236fb423be60eaedfb98b3dcd67: Status 404 returned error can't find the container with id 8f531d5dc2afe9dd137caf5f8fec165618627236fb423be60eaedfb98b3dcd67 Jul 07 00:13:15.180218 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:15.180179 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" event={"ID":"030205c0-753a-4818-b123-49fa19dc30f3","Type":"ContainerStarted","Data":"35485bd8835cc0d9385bd744d827485e63e5427bd5e31c323331d1d6acb29fcb"} Jul 07 00:13:15.180218 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:15.180220 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" event={"ID":"030205c0-753a-4818-b123-49fa19dc30f3","Type":"ContainerStarted","Data":"8f531d5dc2afe9dd137caf5f8fec165618627236fb423be60eaedfb98b3dcd67"} Jul 07 00:13:15.199850 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:15.199798 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" podStartSLOduration=1.1997840929999999 podStartE2EDuration="1.199784093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:13:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:13:15.198374379 +0000 UTC m=+1824.090563057" watchObservedRunningTime="2026-07-07 00:13:15.199784093 +0000 UTC m=+1824.091972813" Jul 07 00:13:15.850739 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:15.850707 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:15.948913 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:15.948886 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:23.525456 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:23.525424 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:23.625893 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:23.625863 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:24.454362 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:24.454327 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:24.454539 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:24.454379 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:24.466885 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:24.466863 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:25.238504 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:25.238479 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:13:33.369076 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:33.369047 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:42.787843 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:42.787807 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:54.201030 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:54.201000 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd"] Jul 07 00:13:54.230376 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:54.230341 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:55.165144 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:55.165083 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:55.322537 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:55.322509 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:55.733495 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:55.733465 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:55.834109 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:55.834033 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:13:55.974757 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:13:55.974729 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:00.825172 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:00.825133 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:09.221648 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.221606 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" podUID="030205c0-753a-4818-b123-49fa19dc30f3" containerName="main" containerID="cri-o://35485bd8835cc0d9385bd744d827485e63e5427bd5e31c323331d1d6acb29fcb" gracePeriod=45 Jul 07 00:14:09.433067 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.433033 2560 generic.go:356] "Generic (PLEG): container finished" podID="030205c0-753a-4818-b123-49fa19dc30f3" containerID="35485bd8835cc0d9385bd744d827485e63e5427bd5e31c323331d1d6acb29fcb" exitCode=0 Jul 07 00:14:09.433203 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.433099 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" event={"ID":"030205c0-753a-4818-b123-49fa19dc30f3","Type":"ContainerDied","Data":"35485bd8835cc0d9385bd744d827485e63e5427bd5e31c323331d1d6acb29fcb"} Jul 07 00:14:09.469049 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.469030 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:14:09.586687 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.586631 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/030205c0-753a-4818-b123-49fa19dc30f3-tls-certs\") pod \"030205c0-753a-4818-b123-49fa19dc30f3\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " Jul 07 00:14:09.586687 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.586673 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-dshm\") pod \"030205c0-753a-4818-b123-49fa19dc30f3\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " Jul 07 00:14:09.586844 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.586723 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-home\") pod \"030205c0-753a-4818-b123-49fa19dc30f3\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " Jul 07 00:14:09.586844 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.586742 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-tmp-dir\") pod \"030205c0-753a-4818-b123-49fa19dc30f3\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " Jul 07 00:14:09.586844 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.586786 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7wps\" (UniqueName: \"kubernetes.io/projected/030205c0-753a-4818-b123-49fa19dc30f3-kube-api-access-g7wps\") pod \"030205c0-753a-4818-b123-49fa19dc30f3\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " Jul 07 00:14:09.586844 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.586829 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-model-cache\") pod \"030205c0-753a-4818-b123-49fa19dc30f3\" (UID: \"030205c0-753a-4818-b123-49fa19dc30f3\") " Jul 07 00:14:09.587040 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.586957 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-home" (OuterVolumeSpecName: "home") pod "030205c0-753a-4818-b123-49fa19dc30f3" (UID: "030205c0-753a-4818-b123-49fa19dc30f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:14:09.587040 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.587011 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "030205c0-753a-4818-b123-49fa19dc30f3" (UID: "030205c0-753a-4818-b123-49fa19dc30f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:14:09.587181 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.587157 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-model-cache" (OuterVolumeSpecName: "model-cache") pod "030205c0-753a-4818-b123-49fa19dc30f3" (UID: "030205c0-753a-4818-b123-49fa19dc30f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:14:09.587228 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.587181 2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-home\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:14:09.587228 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.587196 2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-tmp-dir\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:14:09.589084 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.589064 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-dshm" (OuterVolumeSpecName: "dshm") pod "030205c0-753a-4818-b123-49fa19dc30f3" (UID: "030205c0-753a-4818-b123-49fa19dc30f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:14:09.589189 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.589168 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/030205c0-753a-4818-b123-49fa19dc30f3-kube-api-access-g7wps" (OuterVolumeSpecName: "kube-api-access-g7wps") pod "030205c0-753a-4818-b123-49fa19dc30f3" (UID: "030205c0-753a-4818-b123-49fa19dc30f3"). InnerVolumeSpecName "kube-api-access-g7wps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:14:09.589237 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.589195 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/030205c0-753a-4818-b123-49fa19dc30f3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "030205c0-753a-4818-b123-49fa19dc30f3" (UID: "030205c0-753a-4818-b123-49fa19dc30f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:14:09.688140 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.688115 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7wps\" (UniqueName: \"kubernetes.io/projected/030205c0-753a-4818-b123-49fa19dc30f3-kube-api-access-g7wps\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:14:09.688140 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.688135 2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-model-cache\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:14:09.688403 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.688147 2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/030205c0-753a-4818-b123-49fa19dc30f3-tls-certs\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:14:09.688403 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:09.688155 2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/030205c0-753a-4818-b123-49fa19dc30f3-dshm\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:14:10.439668 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:10.439634 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" event={"ID":"030205c0-753a-4818-b123-49fa19dc30f3","Type":"ContainerDied","Data":"8f531d5dc2afe9dd137caf5f8fec165618627236fb423be60eaedfb98b3dcd67"} Jul 07 00:14:10.440061 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:10.439681 2560 scope.go:117] "RemoveContainer" containerID="35485bd8835cc0d9385bd744d827485e63e5427bd5e31c323331d1d6acb29fcb" Jul 07 00:14:10.440061 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:10.439682 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd" Jul 07 00:14:10.462309 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:10.462285 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd"] Jul 07 00:14:10.466018 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:10.465980 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-02e2a5/test-model-02e2a5-kserve-58776b5b6c-fs9bd"] Jul 07 00:14:11.563092 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:11.563059 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="030205c0-753a-4818-b123-49fa19dc30f3" path="/var/lib/kubelet/pods/030205c0-753a-4818-b123-49fa19dc30f3/volumes" Jul 07 00:14:37.766184 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:37.766140 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:38.042894 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:38.042814 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:38.276678 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:38.276652 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:38.462375 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:38.462339 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:38.936772 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:38.936695 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:39.037314 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:39.037283 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:39.138429 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:39.138397 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:39.238636 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:39.238610 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:45.000422 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:45.000388 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:45.101121 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:45.101092 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:45.289730 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:45.289662 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:45.523313 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:45.523273 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:45.876444 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:45.876414 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:46.198239 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:46.198162 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:46.394966 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:46.394932 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:48.862719 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:48.862684 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:48.951116 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:48.951088 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:49.029331 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:49.029300 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:49.182259 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:49.182202 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:49.622015 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:49.621988 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:49.726600 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:49.726561 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:53.434140 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:53.434112 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:53.915604 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:53.915551 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:54.240442 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:54.240407 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:55.026145 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:55.026114 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:56.956034 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:56.955991 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:57.049245 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:57.049210 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:57.118131 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:57.118104 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:57.219103 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:57.219038 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:57.506232 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:57.506135 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:14:57.606038 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:14:57.605996 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:01.485712 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:01.485677 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:07.510670 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:07.510638 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:08.175210 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:08.175174 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:09.964758 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:09.964724 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:10.331068 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:10.331000 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:10.431442 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:10.431409 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:10.531651 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:10.531622 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:10.607202 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:10.607125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:16.145112 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:16.145077 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:16.208647 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:16.208617 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:16.485069 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:16.485035 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:16.692814 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:16.692783 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:16.855037 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:16.854963 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:17.282592 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:17.282545 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:17.381986 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:17.381955 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:17.537410 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:17.537329 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:22.861119 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:22.861082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:22.917130 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:22.917102 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:23.116328 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:23.116254 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:23.344956 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:23.344923 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:23.690389 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:23.690359 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:24.009069 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:24.009039 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:24.107303 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:24.107274 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:27.266331 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:27.266297 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:27.352994 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:27.352965 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:27.413616 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:27.413565 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:27.719032 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:27.719004 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:27.841129 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:27.841105 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:27.941907 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:27.941879 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:31.697736 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:31.697709 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:32.041883 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:32.041853 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:32.492694 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:32.492661 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:32.814142 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:32.814055 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:35.206443 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:35.206405 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:35.308458 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:35.308427 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:35.522062 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:35.521981 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:35.722441 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:35.722410 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:35.934456 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:35.934370 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:36.034732 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:36.034708 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:37.041078 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:37.041048 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:37.141752 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:37.141725 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:37.790860 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:37.790829 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:37.891218 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:37.891191 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:39.712498 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:39.712469 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:48.972277 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:48.972204 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:49.422701 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:49.422624 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:49.525998 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:49.525974 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:56.722136 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:56.722099 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:15:56.822837 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:15:56.822809 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:17.942455 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.942420 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g"] Jul 07 00:16:17.942945 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.942838 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="030205c0-753a-4818-b123-49fa19dc30f3" containerName="main" Jul 07 00:16:17.942945 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.942851 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="030205c0-753a-4818-b123-49fa19dc30f3" containerName="main" Jul 07 00:16:17.942945 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.942930 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="030205c0-753a-4818-b123-49fa19dc30f3" containerName="main" Jul 07 00:16:17.946118 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.946102 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:17.949240 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.949214 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-72fa7c\"/\"auth-test-model-72fa7c-kserve-self-signed-certs\"" Jul 07 00:16:17.949450 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.949244 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-72fa7c\"/\"kube-root-ca.crt\"" Jul 07 00:16:17.949450 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.949223 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-72fa7c\"/\"default-dockercfg-jjd4n\"" Jul 07 00:16:17.949450 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.949219 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-72fa7c\"/\"openshift-service-ca.crt\"" Jul 07 00:16:17.955410 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.955387 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g"] Jul 07 00:16:17.957452 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.957433 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-model-cache\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:17.957552 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.957470 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tmp-dir\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:17.957552 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.957522 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-home\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:17.957552 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.957542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-dshm\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:17.957680 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.957607 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tls-certs\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:17.957680 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.957654 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22nlr\" (UniqueName: \"kubernetes.io/projected/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-kube-api-access-22nlr\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:17.962387 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:17.962369 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:16:18.058484 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.058464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-model-cache\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.058622 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.058494 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tmp-dir\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.058622 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.058519 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-home\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.058699 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.058645 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-dshm\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.058699 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.058685 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tls-certs\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.058850 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.058713 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22nlr\" (UniqueName: \"kubernetes.io/projected/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-kube-api-access-22nlr\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.058904 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.058884 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-home\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.058959 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.058915 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tmp-dir\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.059019 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.058988 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-model-cache\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.061014 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.060996 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-dshm\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.061295 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.061276 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tls-certs\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.066641 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.066623 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22nlr\" (UniqueName: \"kubernetes.io/projected/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-kube-api-access-22nlr\") pod \"auth-test-model-72fa7c-kserve-59c79457f5-ssh5g\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.256838 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.256812 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:18.385606 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:18.385582 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g"] Jul 07 00:16:18.387936 ip-10-0-135-66 kubenswrapper[2560]: W0707 00:16:18.387911 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6d3da01_ff09_4d22_a0d4_ebc4d229bea2.slice/crio-f85bebbdb368f85d7ca1a2224bb3a0de305f05cc2c60c4e7278ffd43fd4acce6 WatchSource:0}: Error finding container f85bebbdb368f85d7ca1a2224bb3a0de305f05cc2c60c4e7278ffd43fd4acce6: Status 404 returned error can't find the container with id f85bebbdb368f85d7ca1a2224bb3a0de305f05cc2c60c4e7278ffd43fd4acce6 Jul 07 00:16:19.027677 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:19.027642 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" event={"ID":"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2","Type":"ContainerStarted","Data":"1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc"} Jul 07 00:16:19.027677 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:19.027682 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" event={"ID":"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2","Type":"ContainerStarted","Data":"f85bebbdb368f85d7ca1a2224bb3a0de305f05cc2c60c4e7278ffd43fd4acce6"} Jul 07 00:16:19.046485 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:19.046418 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" podStartSLOduration=2.046402585 podStartE2EDuration="2.046402585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:16:19.044784798 +0000 UTC m=+2007.936973477" watchObservedRunningTime="2026-07-07 00:16:19.046402585 +0000 UTC m=+2007.938591262" Jul 07 00:16:28.257731 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:28.257698 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:28.258198 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:28.257781 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:28.269808 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:28.269787 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:16:29.091370 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:16:29.091342 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:18:13.979887 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:13.979847 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:14.715231 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:14.715202 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g"] Jul 07 00:18:14.737706 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:14.737651 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:29.736288 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:29.736247 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" podUID="f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" containerName="main" containerID="cri-o://1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc" gracePeriod=45 Jul 07 00:18:29.982669 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:29.982646 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:18:30.014154 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014089 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-home\") pod \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " Jul 07 00:18:30.014154 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014122 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-dshm\") pod \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " Jul 07 00:18:30.014306 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014171 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tls-certs\") pod \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " Jul 07 00:18:30.014306 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014206 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-model-cache\") pod \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " Jul 07 00:18:30.014306 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014268 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22nlr\" (UniqueName: \"kubernetes.io/projected/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-kube-api-access-22nlr\") pod \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " Jul 07 00:18:30.014306 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014288 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tmp-dir\") pod \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\" (UID: \"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2\") " Jul 07 00:18:30.014503 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014381 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-home" (OuterVolumeSpecName: "home") pod "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" (UID: "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:18:30.014613 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014583 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-model-cache" (OuterVolumeSpecName: "model-cache") pod "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" (UID: "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:18:30.014613 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014599 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" (UID: "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:18:30.014725 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014679 2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-model-cache\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:18:30.014725 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014692 2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tmp-dir\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:18:30.014725 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.014701 2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-home\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:18:30.016562 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.016533 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-kube-api-access-22nlr" (OuterVolumeSpecName: "kube-api-access-22nlr") pod "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" (UID: "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2"). InnerVolumeSpecName "kube-api-access-22nlr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:18:30.016711 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.016624 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" (UID: "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:18:30.016797 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.016777 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-dshm" (OuterVolumeSpecName: "dshm") pod "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" (UID: "f6d3da01-ff09-4d22-a0d4-ebc4d229bea2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:18:30.115367 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.115329 2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-dshm\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:18:30.115367 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.115359 2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-tls-certs\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:18:30.115609 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.115372 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22nlr\" (UniqueName: \"kubernetes.io/projected/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2-kube-api-access-22nlr\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:18:30.643931 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.643899 2560 generic.go:356] "Generic (PLEG): container finished" podID="f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" containerID="1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc" exitCode=0 Jul 07 00:18:30.644113 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.643946 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" Jul 07 00:18:30.644113 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.643988 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" event={"ID":"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2","Type":"ContainerDied","Data":"1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc"} Jul 07 00:18:30.644113 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.644030 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g" event={"ID":"f6d3da01-ff09-4d22-a0d4-ebc4d229bea2","Type":"ContainerDied","Data":"f85bebbdb368f85d7ca1a2224bb3a0de305f05cc2c60c4e7278ffd43fd4acce6"} Jul 07 00:18:30.644113 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.644055 2560 scope.go:117] "RemoveContainer" containerID="1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc" Jul 07 00:18:30.652984 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.652967 2560 scope.go:117] "RemoveContainer" containerID="1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc" Jul 07 00:18:30.653248 ip-10-0-135-66 kubenswrapper[2560]: E0707 00:18:30.653229 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc\": container with ID starting with 1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc not found: ID does not exist" containerID="1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc" Jul 07 00:18:30.653294 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.653258 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc"} err="failed to get container status \"1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc\": rpc error: code = NotFound desc = could not find container \"1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc\": container with ID starting with 1f6ec086193e076919d4670cfcbbd0ad61ab5f86c6070927f82f4460825493fc not found: ID does not exist" Jul 07 00:18:30.677530 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.677510 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g"] Jul 07 00:18:30.694326 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.694305 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/auth-test-model-72fa7c-kserve-59c79457f5-ssh5g"] Jul 07 00:18:30.949991 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.949918 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst"] Jul 07 00:18:30.950343 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.950277 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" containerName="main" Jul 07 00:18:30.950343 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.950287 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" containerName="main" Jul 07 00:18:30.950424 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.950361 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" containerName="main" Jul 07 00:18:30.954916 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.954897 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:30.958074 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.958048 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-aedad1\"/\"openshift-service-ca.crt\"" Jul 07 00:18:30.958202 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.958132 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-aedad1\"/\"kube-root-ca.crt\"" Jul 07 00:18:30.958280 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.958263 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aedad1\"/\"default-dockercfg-v94q6\"" Jul 07 00:18:30.958391 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.958375 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aedad1\"/\"sub-test-model-aedad1-kserve-self-signed-certs\"" Jul 07 00:18:30.965697 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.965677 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst"] Jul 07 00:18:30.972259 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:30.972232 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:18:31.021174 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.021148 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-dshm\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.021276 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.021202 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62650ba0-f669-433e-9eef-ec465ef8fb2c-tls-certs\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.021276 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.021219 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-tmp-dir\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.021276 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.021236 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-home\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.021276 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.021269 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nz28\" (UniqueName: \"kubernetes.io/projected/62650ba0-f669-433e-9eef-ec465ef8fb2c-kube-api-access-2nz28\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.021437 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.021287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-model-cache\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.122119 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.122093 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62650ba0-f669-433e-9eef-ec465ef8fb2c-tls-certs\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.122245 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.122125 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-tmp-dir\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.122245 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.122155 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-home\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.122245 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.122206 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nz28\" (UniqueName: \"kubernetes.io/projected/62650ba0-f669-433e-9eef-ec465ef8fb2c-kube-api-access-2nz28\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.122245 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.122233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-model-cache\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.122428 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.122281 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-dshm\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.122562 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.122538 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-tmp-dir\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.122562 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.122554 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-home\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.122789 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.122710 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-model-cache\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.124564 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.124541 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-dshm\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.124893 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.124876 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62650ba0-f669-433e-9eef-ec465ef8fb2c-tls-certs\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.130186 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.130163 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nz28\" (UniqueName: \"kubernetes.io/projected/62650ba0-f669-433e-9eef-ec465ef8fb2c-kube-api-access-2nz28\") pod \"sub-test-model-aedad1-kserve-675977c5bc-7hfst\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.264039 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.264016 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:31.394192 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.394161 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst"] Jul 07 00:18:31.396428 ip-10-0-135-66 kubenswrapper[2560]: W0707 00:18:31.396404 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62650ba0_f669_433e_9eef_ec465ef8fb2c.slice/crio-31bcfe0cc50535225ab0daf5fcf0cdaa2f2434566e6c97f1ec6ef730beb0f1be WatchSource:0}: Error finding container 31bcfe0cc50535225ab0daf5fcf0cdaa2f2434566e6c97f1ec6ef730beb0f1be: Status 404 returned error can't find the container with id 31bcfe0cc50535225ab0daf5fcf0cdaa2f2434566e6c97f1ec6ef730beb0f1be Jul 07 00:18:31.563736 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.563662 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6d3da01-ff09-4d22-a0d4-ebc4d229bea2" path="/var/lib/kubelet/pods/f6d3da01-ff09-4d22-a0d4-ebc4d229bea2/volumes" Jul 07 00:18:31.649669 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.649627 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" event={"ID":"62650ba0-f669-433e-9eef-ec465ef8fb2c","Type":"ContainerStarted","Data":"c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3"} Jul 07 00:18:31.649669 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.649675 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" event={"ID":"62650ba0-f669-433e-9eef-ec465ef8fb2c","Type":"ContainerStarted","Data":"31bcfe0cc50535225ab0daf5fcf0cdaa2f2434566e6c97f1ec6ef730beb0f1be"} Jul 07 00:18:31.669431 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:31.669387 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" podStartSLOduration=1.66937271 podStartE2EDuration="1.66937271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:18:31.667717348 +0000 UTC m=+2140.559906040" watchObservedRunningTime="2026-07-07 00:18:31.66937271 +0000 UTC m=+2140.561561388" Jul 07 00:18:41.264599 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:41.264551 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:41.264964 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:41.264617 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:41.277052 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:41.277029 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:41.708321 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:41.708266 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:18:59.195628 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:18:59.195590 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:19:34.363829 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:19:34.363797 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:19:57.281401 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:19:57.281362 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:19:57.835334 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:19:57.835306 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst"] Jul 07 00:19:57.864747 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:19:57.864718 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:19:58.779284 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:19:58.779257 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:04.493020 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:04.492977 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:12.855526 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:12.855437 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" podUID="62650ba0-f669-433e-9eef-ec465ef8fb2c" containerName="main" containerID="cri-o://c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3" gracePeriod=45 Jul 07 00:20:13.098025 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.098001 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:20:13.128067 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.127993 2560 generic.go:356] "Generic (PLEG): container finished" podID="62650ba0-f669-433e-9eef-ec465ef8fb2c" containerID="c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3" exitCode=0 Jul 07 00:20:13.128067 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.128037 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" Jul 07 00:20:13.128245 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.128060 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" event={"ID":"62650ba0-f669-433e-9eef-ec465ef8fb2c","Type":"ContainerDied","Data":"c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3"} Jul 07 00:20:13.128245 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.128111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst" event={"ID":"62650ba0-f669-433e-9eef-ec465ef8fb2c","Type":"ContainerDied","Data":"31bcfe0cc50535225ab0daf5fcf0cdaa2f2434566e6c97f1ec6ef730beb0f1be"} Jul 07 00:20:13.128245 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.128137 2560 scope.go:117] "RemoveContainer" containerID="c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3" Jul 07 00:20:13.138061 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.138040 2560 scope.go:117] "RemoveContainer" containerID="c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3" Jul 07 00:20:13.138342 ip-10-0-135-66 kubenswrapper[2560]: E0707 00:20:13.138323 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3\": container with ID starting with c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3 not found: ID does not exist" containerID="c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3" Jul 07 00:20:13.138414 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.138351 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3"} err="failed to get container status \"c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3\": rpc error: code = NotFound desc = could not find container \"c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3\": container with ID starting with c1177dda102500da3c400222f956d09bb9daa634c3e9bed31a159326b779c6d3 not found: ID does not exist" Jul 07 00:20:13.183367 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183347 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nz28\" (UniqueName: \"kubernetes.io/projected/62650ba0-f669-433e-9eef-ec465ef8fb2c-kube-api-access-2nz28\") pod \"62650ba0-f669-433e-9eef-ec465ef8fb2c\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " Jul 07 00:20:13.183450 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183395 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62650ba0-f669-433e-9eef-ec465ef8fb2c-tls-certs\") pod \"62650ba0-f669-433e-9eef-ec465ef8fb2c\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " Jul 07 00:20:13.183450 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183430 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-tmp-dir\") pod \"62650ba0-f669-433e-9eef-ec465ef8fb2c\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " Jul 07 00:20:13.183450 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183449 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-home\") pod \"62650ba0-f669-433e-9eef-ec465ef8fb2c\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " Jul 07 00:20:13.183561 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183470 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-model-cache\") pod \"62650ba0-f669-433e-9eef-ec465ef8fb2c\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " Jul 07 00:20:13.183561 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183492 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-dshm\") pod \"62650ba0-f669-433e-9eef-ec465ef8fb2c\" (UID: \"62650ba0-f669-433e-9eef-ec465ef8fb2c\") " Jul 07 00:20:13.183780 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183732 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "62650ba0-f669-433e-9eef-ec465ef8fb2c" (UID: "62650ba0-f669-433e-9eef-ec465ef8fb2c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:13.183780 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183752 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-home" (OuterVolumeSpecName: "home") pod "62650ba0-f669-433e-9eef-ec465ef8fb2c" (UID: "62650ba0-f669-433e-9eef-ec465ef8fb2c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:13.183780 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183764 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-model-cache" (OuterVolumeSpecName: "model-cache") pod "62650ba0-f669-433e-9eef-ec465ef8fb2c" (UID: "62650ba0-f669-433e-9eef-ec465ef8fb2c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:13.183962 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183833 2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-tmp-dir\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:20:13.183962 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183843 2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-home\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:20:13.183962 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.183852 2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-model-cache\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:20:13.185689 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.185670 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62650ba0-f669-433e-9eef-ec465ef8fb2c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "62650ba0-f669-433e-9eef-ec465ef8fb2c" (UID: "62650ba0-f669-433e-9eef-ec465ef8fb2c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:20:13.185689 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.185677 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-dshm" (OuterVolumeSpecName: "dshm") pod "62650ba0-f669-433e-9eef-ec465ef8fb2c" (UID: "62650ba0-f669-433e-9eef-ec465ef8fb2c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:13.185789 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.185742 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62650ba0-f669-433e-9eef-ec465ef8fb2c-kube-api-access-2nz28" (OuterVolumeSpecName: "kube-api-access-2nz28") pod "62650ba0-f669-433e-9eef-ec465ef8fb2c" (UID: "62650ba0-f669-433e-9eef-ec465ef8fb2c"). InnerVolumeSpecName "kube-api-access-2nz28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:20:13.284468 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.284442 2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62650ba0-f669-433e-9eef-ec465ef8fb2c-tls-certs\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:20:13.284468 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.284468 2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62650ba0-f669-433e-9eef-ec465ef8fb2c-dshm\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:20:13.284619 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.284482 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nz28\" (UniqueName: \"kubernetes.io/projected/62650ba0-f669-433e-9eef-ec465ef8fb2c-kube-api-access-2nz28\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:20:13.455306 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.455274 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst"] Jul 07 00:20:13.462175 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.462150 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-aedad1/sub-test-model-aedad1-kserve-675977c5bc-7hfst"] Jul 07 00:20:13.562527 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:13.562499 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62650ba0-f669-433e-9eef-ec465ef8fb2c" path="/var/lib/kubelet/pods/62650ba0-f669-433e-9eef-ec465ef8fb2c/volumes" Jul 07 00:20:26.544015 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.543982 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx"] Jul 07 00:20:26.544393 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.544337 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62650ba0-f669-433e-9eef-ec465ef8fb2c" containerName="main" Jul 07 00:20:26.544393 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.544347 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="62650ba0-f669-433e-9eef-ec465ef8fb2c" containerName="main" Jul 07 00:20:26.544480 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.544416 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="62650ba0-f669-433e-9eef-ec465ef8fb2c" containerName="main" Jul 07 00:20:26.547443 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.547427 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.551638 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.551611 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-72fa7c\"/\"default-dockercfg-jjd4n\"" Jul 07 00:20:26.551773 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.551698 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-72fa7c\"/\"openshift-service-ca.crt\"" Jul 07 00:20:26.551773 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.551752 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-72fa7c\"/\"kube-root-ca.crt\"" Jul 07 00:20:26.551905 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.551838 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-72fa7c\"/\"rate-test-model-72fa7c-kserve-self-signed-certs\"" Jul 07 00:20:26.557991 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.557966 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx"] Jul 07 00:20:26.586207 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.586182 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:26.599228 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.599200 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-tmp-dir\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.599331 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.599312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtjnz\" (UniqueName: \"kubernetes.io/projected/055c24bd-989a-46dc-a20b-cc6232adf304-kube-api-access-dtjnz\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.599389 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.599361 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-model-cache\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.599447 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.599391 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-home\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.599447 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.599413 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-dshm\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.599560 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.599505 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/055c24bd-989a-46dc-a20b-cc6232adf304-tls-certs\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.700380 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.700358 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/055c24bd-989a-46dc-a20b-cc6232adf304-tls-certs\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.700478 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.700392 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-tmp-dir\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.700540 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.700524 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtjnz\" (UniqueName: \"kubernetes.io/projected/055c24bd-989a-46dc-a20b-cc6232adf304-kube-api-access-dtjnz\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.700636 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.700600 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-model-cache\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.700696 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.700640 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-home\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.700696 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.700661 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-dshm\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.700801 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.700744 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-tmp-dir\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.700915 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.700897 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-model-cache\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.701035 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.700948 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-home\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.702872 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.702852 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-dshm\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.703055 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.703037 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/055c24bd-989a-46dc-a20b-cc6232adf304-tls-certs\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.710466 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.710448 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtjnz\" (UniqueName: \"kubernetes.io/projected/055c24bd-989a-46dc-a20b-cc6232adf304-kube-api-access-dtjnz\") pod \"rate-test-model-72fa7c-kserve-5cc7575c55-zrghx\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:26.859000 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:26.858924 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:27.008087 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:27.008022 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx"] Jul 07 00:20:27.010436 ip-10-0-135-66 kubenswrapper[2560]: W0707 00:20:27.010406 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod055c24bd_989a_46dc_a20b_cc6232adf304.slice/crio-b6afe2ab81f6a4cf0febc8ec1b6f2a702db91e5ef1944e38251347118ca3c2a5 WatchSource:0}: Error finding container b6afe2ab81f6a4cf0febc8ec1b6f2a702db91e5ef1944e38251347118ca3c2a5: Status 404 returned error can't find the container with id b6afe2ab81f6a4cf0febc8ec1b6f2a702db91e5ef1944e38251347118ca3c2a5 Jul 07 00:20:27.195812 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:27.195697 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" event={"ID":"055c24bd-989a-46dc-a20b-cc6232adf304","Type":"ContainerStarted","Data":"4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2"} Jul 07 00:20:27.195812 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:27.195745 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" event={"ID":"055c24bd-989a-46dc-a20b-cc6232adf304","Type":"ContainerStarted","Data":"b6afe2ab81f6a4cf0febc8ec1b6f2a702db91e5ef1944e38251347118ca3c2a5"} Jul 07 00:20:27.217030 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:27.216978 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" podStartSLOduration=1.2169635 podStartE2EDuration="1.2169635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:20:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:20:27.214462819 +0000 UTC m=+2256.106651496" watchObservedRunningTime="2026-07-07 00:20:27.2169635 +0000 UTC m=+2256.109152177" Jul 07 00:20:36.859768 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:36.859719 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:36.860273 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:36.859789 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:36.872921 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:36.872888 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:37.254039 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:37.254010 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:20:37.870487 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:37.870447 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:37.968952 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:37.968895 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:43.540801 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:43.540762 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:50.134310 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:50.134257 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:50.247460 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:50.247424 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:57.685224 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:57.685185 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:20:57.780346 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:20:57.780320 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:10.029488 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:10.029457 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:10.859105 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:10.859065 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx"] Jul 07 00:21:10.883950 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:10.883924 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:11.980915 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:11.980884 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:12.415521 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:12.415430 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:12.527582 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:12.527537 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:12.747200 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:12.747173 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:17.493529 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:17.493493 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:25.879253 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:25.879215 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" podUID="055c24bd-989a-46dc-a20b-cc6232adf304" containerName="main" containerID="cri-o://4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2" gracePeriod=45 Jul 07 00:21:26.137643 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.137588 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:21:26.189846 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.189824 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-dshm\") pod \"055c24bd-989a-46dc-a20b-cc6232adf304\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " Jul 07 00:21:26.189984 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.189887 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtjnz\" (UniqueName: \"kubernetes.io/projected/055c24bd-989a-46dc-a20b-cc6232adf304-kube-api-access-dtjnz\") pod \"055c24bd-989a-46dc-a20b-cc6232adf304\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " Jul 07 00:21:26.189984 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.189905 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-model-cache\") pod \"055c24bd-989a-46dc-a20b-cc6232adf304\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " Jul 07 00:21:26.189984 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.189921 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-home\") pod \"055c24bd-989a-46dc-a20b-cc6232adf304\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " Jul 07 00:21:26.190161 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.189991 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-tmp-dir\") pod \"055c24bd-989a-46dc-a20b-cc6232adf304\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " Jul 07 00:21:26.190161 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.190016 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/055c24bd-989a-46dc-a20b-cc6232adf304-tls-certs\") pod \"055c24bd-989a-46dc-a20b-cc6232adf304\" (UID: \"055c24bd-989a-46dc-a20b-cc6232adf304\") " Jul 07 00:21:26.190263 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.190175 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-model-cache" (OuterVolumeSpecName: "model-cache") pod "055c24bd-989a-46dc-a20b-cc6232adf304" (UID: "055c24bd-989a-46dc-a20b-cc6232adf304"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:21:26.190309 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.190248 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-home" (OuterVolumeSpecName: "home") pod "055c24bd-989a-46dc-a20b-cc6232adf304" (UID: "055c24bd-989a-46dc-a20b-cc6232adf304"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:21:26.190309 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.190267 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "055c24bd-989a-46dc-a20b-cc6232adf304" (UID: "055c24bd-989a-46dc-a20b-cc6232adf304"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:21:26.190387 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.190367 2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-model-cache\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:21:26.190421 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.190386 2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-home\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:21:26.190455 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.190420 2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-tmp-dir\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:21:26.192039 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.192009 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/055c24bd-989a-46dc-a20b-cc6232adf304-kube-api-access-dtjnz" (OuterVolumeSpecName: "kube-api-access-dtjnz") pod "055c24bd-989a-46dc-a20b-cc6232adf304" (UID: "055c24bd-989a-46dc-a20b-cc6232adf304"). InnerVolumeSpecName "kube-api-access-dtjnz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:21:26.192182 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.192162 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/055c24bd-989a-46dc-a20b-cc6232adf304-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "055c24bd-989a-46dc-a20b-cc6232adf304" (UID: "055c24bd-989a-46dc-a20b-cc6232adf304"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:21:26.192226 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.192184 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-dshm" (OuterVolumeSpecName: "dshm") pod "055c24bd-989a-46dc-a20b-cc6232adf304" (UID: "055c24bd-989a-46dc-a20b-cc6232adf304"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:21:26.291042 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.291020 2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/055c24bd-989a-46dc-a20b-cc6232adf304-dshm\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:21:26.291042 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.291040 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtjnz\" (UniqueName: \"kubernetes.io/projected/055c24bd-989a-46dc-a20b-cc6232adf304-kube-api-access-dtjnz\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:21:26.291164 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.291050 2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/055c24bd-989a-46dc-a20b-cc6232adf304-tls-certs\") on node \"ip-10-0-135-66.ec2.internal\" DevicePath \"\"" Jul 07 00:21:26.477992 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.477968 2560 generic.go:356] "Generic (PLEG): container finished" podID="055c24bd-989a-46dc-a20b-cc6232adf304" containerID="4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2" exitCode=0 Jul 07 00:21:26.478095 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.478008 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" Jul 07 00:21:26.478095 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.478034 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" event={"ID":"055c24bd-989a-46dc-a20b-cc6232adf304","Type":"ContainerDied","Data":"4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2"} Jul 07 00:21:26.478095 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.478065 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx" event={"ID":"055c24bd-989a-46dc-a20b-cc6232adf304","Type":"ContainerDied","Data":"b6afe2ab81f6a4cf0febc8ec1b6f2a702db91e5ef1944e38251347118ca3c2a5"} Jul 07 00:21:26.478095 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.478081 2560 scope.go:117] "RemoveContainer" containerID="4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2" Jul 07 00:21:26.487082 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.487066 2560 scope.go:117] "RemoveContainer" containerID="4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2" Jul 07 00:21:26.487313 ip-10-0-135-66 kubenswrapper[2560]: E0707 00:21:26.487298 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2\": container with ID starting with 4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2 not found: ID does not exist" containerID="4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2" Jul 07 00:21:26.487381 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.487320 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2"} err="failed to get container status \"4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2\": rpc error: code = NotFound desc = could not find container \"4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2\": container with ID starting with 4ecd67afa388d729c9f92e0e0f1ebc3ce195d1d14282d2759a13bd736c5bd7c2 not found: ID does not exist" Jul 07 00:21:26.499158 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.499133 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx"] Jul 07 00:21:26.503637 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:26.503618 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-72fa7c/rate-test-model-72fa7c-kserve-5cc7575c55-zrghx"] Jul 07 00:21:27.563424 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:27.563388 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="055c24bd-989a-46dc-a20b-cc6232adf304" path="/var/lib/kubelet/pods/055c24bd-989a-46dc-a20b-cc6232adf304/volumes" Jul 07 00:21:29.327793 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:29.327760 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:54.521331 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:54.521291 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:54.796224 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:54.796144 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:55.108531 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:55.108447 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:55.253003 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:55.252970 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:55.669350 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:55.669321 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:55.769104 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:55.769076 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:55.867321 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:55.867293 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:21:55.993561 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:21:55.993530 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:01.608169 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:01.608139 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:01.709621 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:01.709592 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:01.935968 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:01.935893 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:02.148705 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:02.148678 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:02.605356 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:02.605329 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67c9bcb79d-bnxlt_12527d13-c1a8-4a92-a9a2-8d31c6791614/authorino/0.log" Jul 07 00:22:06.448464 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:06.448422 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:06.514011 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:06.513984 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:08.782702 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:08.782670 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bfc4f9689-jcqlv_b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a/manager/0.log" Jul 07 00:22:08.789380 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:08.789358 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bfc4f9689-jcqlv_b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a/copy-manifests/0.log" Jul 07 00:22:10.305430 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:10.305398 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67c9bcb79d-bnxlt_12527d13-c1a8-4a92-a9a2-8d31c6791614/authorino/0.log" Jul 07 00:22:10.558107 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:10.558029 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rp7cx_af9a8e1b-8f6e-4e3f-a27c-95089050849c/manager/0.log" Jul 07 00:22:10.672068 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:10.672046 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8jv5t_371d6297-c9a9-4518-8a3a-27e585b117ba/kuadrant-console-plugin/0.log" Jul 07 00:22:11.180837 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:11.180803 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-tkprt_0df8224e-1240-40a0-837f-4d5ad17ce493/manager/0.log" Jul 07 00:22:11.803589 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:11.803543 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5646546ff9-ctv9q_2d5dc851-5a03-4587-8f36-f5408da0a3c2/kube-auth-proxy/0.log" Jul 07 00:22:12.922689 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:12.922656 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-kvhfb_03c5e60a-6af7-4a21-8631-06c304998c3c/main/0.log" Jul 07 00:22:13.174148 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:13.174072 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbmjmk_1a0ee9fd-5246-4e3d-a6bb-2ee3f457d9b7/main/0.log" Jul 07 00:22:13.300180 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:13.300158 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-9hwmw_fa037dd6-0e65-43d3-9e1c-fee0548b924b/main/0.log" Jul 07 00:22:13.423074 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:13.423049 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-jlfbq_78eaafa9-e153-4a67-b209-a290b85c4d4a/main/0.log" Jul 07 00:22:17.574720 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:17.574683 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:17.672938 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:17.672903 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:20.521109 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:20.521084 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nng68_2a524114-7358-43de-85d9-00e55e281c8a/global-pull-secret-syncer/0.log" Jul 07 00:22:20.581090 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:20.581062 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" Jul 07 00:22:20.700648 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:20.700626 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-66.ec2.internal_33725b36b540b2d810e88a2177f8bbbe/haproxy/0.log" Jul 07 00:22:24.527891 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:24.527858 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:24.587932 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:24.587901 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:26.244719 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:26.244686 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67c9bcb79d-bnxlt_12527d13-c1a8-4a92-a9a2-8d31c6791614/authorino/0.log" Jul 07 00:22:26.301290 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:26.301260 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rp7cx_af9a8e1b-8f6e-4e3f-a27c-95089050849c/manager/0.log" Jul 07 00:22:26.329331 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:26.329311 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8jv5t_371d6297-c9a9-4518-8a3a-27e585b117ba/kuadrant-console-plugin/0.log" Jul 07 00:22:26.643065 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:26.642977 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-tkprt_0df8224e-1240-40a0-837f-4d5ad17ce493/manager/0.log" Jul 07 00:22:28.268299 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.268268 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d999c3e1-d0a7-4d69-96eb-f954ba0dafd5/alertmanager/0.log" Jul 07 00:22:28.290360 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.290332 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d999c3e1-d0a7-4d69-96eb-f954ba0dafd5/config-reloader/0.log" Jul 07 00:22:28.316918 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.316885 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d999c3e1-d0a7-4d69-96eb-f954ba0dafd5/kube-rbac-proxy-web/0.log" Jul 07 00:22:28.337131 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.337112 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d999c3e1-d0a7-4d69-96eb-f954ba0dafd5/kube-rbac-proxy/0.log" Jul 07 00:22:28.361550 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.361489 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d999c3e1-d0a7-4d69-96eb-f954ba0dafd5/kube-rbac-proxy-metric/0.log" Jul 07 00:22:28.384886 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.384866 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d999c3e1-d0a7-4d69-96eb-f954ba0dafd5/prom-label-proxy/0.log" Jul 07 00:22:28.405114 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.405074 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d999c3e1-d0a7-4d69-96eb-f954ba0dafd5/init-config-reloader/0.log" Jul 07 00:22:28.563355 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.563334 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-85b7659487-6965h_d607b3b8-b7c9-40d4-ad63-92d28e7c18b8/metrics-server/0.log" Jul 07 00:22:28.592971 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.592943 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-sbg82_26df84eb-c8c8-439a-91e8-25dc1a6e77e2/monitoring-plugin/0.log" Jul 07 00:22:28.723601 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.723561 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nbl2d_f761b63d-bb73-4e87-890f-cef98fccf41c/node-exporter/0.log" Jul 07 00:22:28.750427 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.750409 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nbl2d_f761b63d-bb73-4e87-890f-cef98fccf41c/kube-rbac-proxy/0.log" Jul 07 00:22:28.773638 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.773622 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nbl2d_f761b63d-bb73-4e87-890f-cef98fccf41c/init-textfile/0.log" Jul 07 00:22:28.883214 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.883193 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fp8mk_d7930878-bc87-4bf7-a713-9863d32929ca/kube-rbac-proxy-main/0.log" Jul 07 00:22:28.909479 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.909455 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fp8mk_d7930878-bc87-4bf7-a713-9863d32929ca/kube-rbac-proxy-self/0.log" Jul 07 00:22:28.930797 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.930781 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fp8mk_d7930878-bc87-4bf7-a713-9863d32929ca/openshift-state-metrics/0.log" Jul 07 00:22:28.969064 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.969036 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b39eccb2-5950-474e-ba58-bddbf5c16334/prometheus/0.log" Jul 07 00:22:28.989162 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:28.989114 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b39eccb2-5950-474e-ba58-bddbf5c16334/config-reloader/0.log" Jul 07 00:22:29.016690 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.016669 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b39eccb2-5950-474e-ba58-bddbf5c16334/thanos-sidecar/0.log" Jul 07 00:22:29.038824 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.038808 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b39eccb2-5950-474e-ba58-bddbf5c16334/kube-rbac-proxy-web/0.log" Jul 07 00:22:29.061592 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.061548 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b39eccb2-5950-474e-ba58-bddbf5c16334/kube-rbac-proxy/0.log" Jul 07 00:22:29.084063 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.084047 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b39eccb2-5950-474e-ba58-bddbf5c16334/kube-rbac-proxy-thanos/0.log" Jul 07 00:22:29.135090 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.135054 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b39eccb2-5950-474e-ba58-bddbf5c16334/init-config-reloader/0.log" Jul 07 00:22:29.308357 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.308295 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-288bx_deef3342-f965-4f87-8a30-8ae7007a8a31/prometheus-operator-admission-webhook/0.log" Jul 07 00:22:29.417519 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.417496 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b77bb6784-dwf8f_dec7b01b-e69f-423b-a48e-6919828a859f/thanos-query/0.log" Jul 07 00:22:29.443388 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.443366 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b77bb6784-dwf8f_dec7b01b-e69f-423b-a48e-6919828a859f/kube-rbac-proxy-web/0.log" Jul 07 00:22:29.477137 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.477117 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b77bb6784-dwf8f_dec7b01b-e69f-423b-a48e-6919828a859f/kube-rbac-proxy/0.log" Jul 07 00:22:29.503144 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.503122 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b77bb6784-dwf8f_dec7b01b-e69f-423b-a48e-6919828a859f/prom-label-proxy/0.log" Jul 07 00:22:29.525769 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.525747 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b77bb6784-dwf8f_dec7b01b-e69f-423b-a48e-6919828a859f/kube-rbac-proxy-rules/0.log" Jul 07 00:22:29.546689 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:29.546671 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b77bb6784-dwf8f_dec7b01b-e69f-423b-a48e-6919828a859f/kube-rbac-proxy-metrics/0.log" Jul 07 00:22:30.501418 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.501390 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4"] Jul 07 00:22:30.501783 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.501761 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="055c24bd-989a-46dc-a20b-cc6232adf304" containerName="main" Jul 07 00:22:30.501783 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.501772 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="055c24bd-989a-46dc-a20b-cc6232adf304" containerName="main" Jul 07 00:22:30.501855 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.501829 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="055c24bd-989a-46dc-a20b-cc6232adf304" containerName="main" Jul 07 00:22:30.504916 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.504900 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.507093 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.507075 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rg28k\"/\"kube-root-ca.crt\"" Jul 07 00:22:30.507901 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.507883 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rg28k\"/\"default-dockercfg-dqkqt\"" Jul 07 00:22:30.507987 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.507887 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rg28k\"/\"openshift-service-ca.crt\"" Jul 07 00:22:30.513449 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.513431 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4"] Jul 07 00:22:30.598411 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.598392 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-sys\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.598526 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.598423 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-lib-modules\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.598526 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.598476 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjxz4\" (UniqueName: \"kubernetes.io/projected/4de67530-37cc-4ed4-9919-94691e25a10c-kube-api-access-kjxz4\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.598674 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.598545 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-podres\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.598674 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.598631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-proc\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.699098 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.699074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjxz4\" (UniqueName: \"kubernetes.io/projected/4de67530-37cc-4ed4-9919-94691e25a10c-kube-api-access-kjxz4\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.699180 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.699104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-podres\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.699180 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.699130 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-proc\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.699180 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.699149 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-sys\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.699180 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.699170 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-lib-modules\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.699322 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.699223 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-proc\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.699322 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.699240 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-sys\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.699322 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.699241 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-podres\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.699322 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.699286 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4de67530-37cc-4ed4-9919-94691e25a10c-lib-modules\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.707833 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.707806 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjxz4\" (UniqueName: \"kubernetes.io/projected/4de67530-37cc-4ed4-9919-94691e25a10c-kube-api-access-kjxz4\") pod \"perf-node-gather-daemonset-m4bx4\" (UID: \"4de67530-37cc-4ed4-9919-94691e25a10c\") " pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.815196 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.815148 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:30.940022 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:30.940002 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4"] Jul 07 00:22:30.942090 ip-10-0-135-66 kubenswrapper[2560]: W0707 00:22:30.941888 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4de67530_37cc_4ed4_9919_94691e25a10c.slice/crio-23f98388d178fea9194132e15461313164716b3d730cbb6bac44daf3511203e2 WatchSource:0}: Error finding container 23f98388d178fea9194132e15461313164716b3d730cbb6bac44daf3511203e2: Status 404 returned error can't find the container with id 23f98388d178fea9194132e15461313164716b3d730cbb6bac44daf3511203e2 Jul 07 00:22:31.764092 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:31.764061 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" event={"ID":"4de67530-37cc-4ed4-9919-94691e25a10c","Type":"ContainerStarted","Data":"2d340f22641cff5441ce226f39561d4742201128324a8b0181b6cc223b33d97e"} Jul 07 00:22:31.764457 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:31.764103 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" event={"ID":"4de67530-37cc-4ed4-9919-94691e25a10c","Type":"ContainerStarted","Data":"23f98388d178fea9194132e15461313164716b3d730cbb6bac44daf3511203e2"} Jul 07 00:22:31.764457 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:31.764177 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:31.779364 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:31.779316 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" podStartSLOduration=1.779302876 podStartE2EDuration="1.779302876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 00:22:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:22:31.778087557 +0000 UTC m=+2380.670276234" watchObservedRunningTime="2026-07-07 00:22:31.779302876 +0000 UTC m=+2380.671491553" Jul 07 00:22:33.392816 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:33.392785 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7d774_029cdfae-970b-4894-957c-8e999be93a46/dns/0.log" Jul 07 00:22:33.428298 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:33.428268 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7d774_029cdfae-970b-4894-957c-8e999be93a46/kube-rbac-proxy/0.log" Jul 07 00:22:33.687837 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:33.687754 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qcg78_1f61dbcd-7f35-4161-8a71-2a737eed2ded/dns-node-resolver/0.log" Jul 07 00:22:34.299173 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:34.299140 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kbskp_454989b3-ca4f-4b8d-8dc0-17f00a0b55c3/node-ca/0.log" Jul 07 00:22:35.454778 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:35.454745 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5646546ff9-ctv9q_2d5dc851-5a03-4587-8f36-f5408da0a3c2/kube-auth-proxy/0.log" Jul 07 00:22:35.962214 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:35.962178 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:36.031843 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:36.031821 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6l7np_a34c708d-ebcf-49e2-ad65-a18ece0d1858/konnectivity-agent/0.log" limit=1024 Jul 07 00:22:36.364336 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:36.364277 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-99g66_e133d6cd-0751-4e21-815b-c29b1f6b7327/serve-healthcheck-canary/0.log" Jul 07 00:22:36.877728 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:36.877702 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7k55q_4200ae94-dad4-44ee-ba8d-421476fcc12b/kube-rbac-proxy/0.log" Jul 07 00:22:36.898772 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:36.898754 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7k55q_4200ae94-dad4-44ee-ba8d-421476fcc12b/exporter/0.log" Jul 07 00:22:36.922662 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:36.922645 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7k55q_4200ae94-dad4-44ee-ba8d-421476fcc12b/extractor/0.log" Jul 07 00:22:37.779085 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:37.779062 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rg28k/perf-node-gather-daemonset-m4bx4" Jul 07 00:22:40.126467 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:40.126437 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bfc4f9689-jcqlv_b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a/manager/0.log" Jul 07 00:22:40.155816 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:40.155795 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bfc4f9689-jcqlv_b8bf98f3-eb4a-4bb3-af21-e57da6d7bf0a/copy-manifests/0.log" Jul 07 00:22:41.539858 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:41.539830 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58544fcc46-ggx4f_9e739d36-6ad3-4689-a779-68f7f566c40c/manager/0.log" Jul 07 00:22:41.593246 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:41.593223 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hdlf9_fb09c2ac-d728-4403-bec3-419aecd38502/openshift-lws-operator/0.log" Jul 07 00:22:41.693934 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:41.693909 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5646546ff9-ctv9q_2d5dc851-5a03-4587-8f36-f5408da0a3c2/kube-auth-proxy/0.log" Jul 07 00:22:43.138368 ip-10-0-135-66 kubenswrapper[2560]: I0707 00:22:43.138330 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5646546ff9-ctv9q_2d5dc851-5a03-4587-8f36-f5408da0a3c2/kube-auth-proxy/0.log"