Jul 10 18:37:59.076025 ip-10-0-137-86 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 18:37:59.076037 ip-10-0-137-86 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 18:37:59.076044 ip-10-0-137-86 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 18:37:59.076264 ip-10-0-137-86 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 18:38:09.204861 ip-10-0-137-86 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 18:38:09.204875 ip-10-0-137-86 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 21734ebb6a0642598e02f36912cdb133 -- Jul 10 18:40:09.592906 ip-10-0-137-86 systemd[1]: Starting Kubernetes Kubelet... Jul 10 18:40:10.055567 ip-10-0-137-86 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:40:10.055567 ip-10-0-137-86 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 18:40:10.055567 ip-10-0-137-86 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:40:10.055567 ip-10-0-137-86 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 18:40:10.055567 ip-10-0-137-86 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:40:10.056642 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.056497 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 18:40:10.065617 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065596 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:40:10.065617 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065614 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:40:10.065617 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065618 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:40:10.065617 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065621 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:40:10.065617 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065625 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065628 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065631 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065634 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065637 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065641 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065644 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065647 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065650 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065653 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065655 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065658 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065676 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065679 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065682 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065685 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065687 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065691 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065696 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:40:10.065850 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065700 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065702 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065705 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065709 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065712 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065715 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065718 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065722 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065727 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065731 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065736 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065740 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065744 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065747 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065751 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065755 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065759 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065763 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065766 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065769 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:40:10.066339 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065772 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065775 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065778 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065780 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065783 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065786 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065788 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065791 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065794 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065797 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065800 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065802 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065807 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065810 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065813 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065818 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065822 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065825 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065827 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:40:10.066854 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065830 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065833 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065836 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065839 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065841 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065844 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065847 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065850 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065853 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065856 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065859 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065862 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065865 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065868 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065871 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065874 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065877 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065879 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065882 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065884 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:40:10.067354 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065887 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065891 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065894 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065897 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065901 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065904 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065907 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065910 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065912 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065915 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065918 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065921 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065924 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065927 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065930 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065932 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.065935 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066693 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066700 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066703 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:40:10.067896 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066707 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066710 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066713 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066716 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066720 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066742 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066747 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066750 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066754 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066757 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066759 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066762 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066765 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066768 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066771 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066775 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066777 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066780 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066783 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066786 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:40:10.068445 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066788 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066791 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066794 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066797 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066801 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066804 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066806 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066809 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066812 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066815 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066817 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066821 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066824 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066827 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066830 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066833 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066836 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066838 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066841 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066843 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:40:10.068968 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066846 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066849 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066852 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066855 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066860 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066864 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066867 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066871 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066875 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066877 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066880 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066883 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066886 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066888 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066891 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066894 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066897 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066900 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066903 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066906 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:40:10.069521 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066909 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066913 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066916 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066919 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066922 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066925 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066928 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066930 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066933 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066935 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066938 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066941 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066944 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066946 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066949 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066952 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066956 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066960 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066963 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:40:10.070060 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066966 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066970 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066973 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066976 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066978 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066982 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066984 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066987 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066990 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.066997 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067000 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067003 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067006 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067008 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067011 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067014 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067017 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067433 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067439 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067443 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:40:10.070555 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067446 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067449 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067452 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067455 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067458 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067461 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067464 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067467 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067469 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067472 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067474 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067477 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067480 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067483 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067488 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067491 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067494 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067497 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067501 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:40:10.071081 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067504 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067507 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067511 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067514 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067517 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067520 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067523 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067526 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067528 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067532 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067535 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067538 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067540 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067543 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067546 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067550 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067554 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067557 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067560 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:40:10.071557 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067563 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067566 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067569 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067572 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067575 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067578 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067580 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067584 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067588 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067591 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067593 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067596 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067599 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067602 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067606 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067611 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067614 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067617 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067620 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067622 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:40:10.072058 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067625 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067628 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067631 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067633 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067636 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067639 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067642 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067644 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067648 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067651 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067654 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067657 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067659 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067662 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067665 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067668 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067670 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067673 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067676 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067679 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:40:10.072599 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067681 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067684 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067687 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067690 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067692 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067695 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067698 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067702 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067705 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067708 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067711 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067713 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067716 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067734 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067737 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067740 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067743 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.067745 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067816 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067823 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 18:40:10.073128 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067830 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067835 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067839 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067843 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067847 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067852 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067858 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067861 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067865 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067868 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067872 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067875 2563 flags.go:64] FLAG: --cgroup-root="" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067878 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067881 2563 flags.go:64] FLAG: --client-ca-file="" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067885 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067888 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067892 2563 flags.go:64] FLAG: --cluster-domain="" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067895 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067899 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067903 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067907 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067911 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067914 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 18:40:10.073679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067917 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067921 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067924 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067927 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067930 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067934 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067937 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067941 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067944 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067947 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067950 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067953 2563 flags.go:64] FLAG: --enable-server="true" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067956 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067961 2563 flags.go:64] FLAG: --event-burst="100" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067964 2563 flags.go:64] FLAG: --event-qps="50" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067969 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067972 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067975 2563 flags.go:64] FLAG: --eviction-hard="" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067979 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067983 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067986 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067989 2563 flags.go:64] FLAG: --eviction-soft="" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067992 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.067995 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068000 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 18:40:10.074316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068003 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068006 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068009 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068013 2563 flags.go:64] FLAG: --feature-gates="" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068016 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068020 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068023 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068026 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068029 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068032 2563 flags.go:64] FLAG: --help="false" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068035 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068039 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068042 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068045 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068048 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068052 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068055 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068058 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068061 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068064 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068067 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068070 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068075 2563 flags.go:64] FLAG: --kube-reserved="" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068078 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 18:40:10.074998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068081 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068084 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068087 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068090 2563 flags.go:64] FLAG: --lock-file="" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068093 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068096 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068099 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068105 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068108 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068111 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068114 2563 flags.go:64] FLAG: --logging-format="text" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068118 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068122 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068125 2563 flags.go:64] FLAG: --manifest-url="" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068128 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068132 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068135 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068139 2563 flags.go:64] FLAG: --max-pods="110" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068142 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068146 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068149 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068152 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068155 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068158 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068161 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 18:40:10.075632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068169 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068173 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068176 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068179 2563 flags.go:64] FLAG: --pod-cidr="" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068182 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068190 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068194 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068197 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068200 2563 flags.go:64] FLAG: --port="10250" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068203 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068206 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-015eeef128e63d9ad" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068210 2563 flags.go:64] FLAG: --qos-reserved="" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068213 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068217 2563 flags.go:64] FLAG: --register-node="true" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068220 2563 flags.go:64] FLAG: --register-with-taints="" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068224 2563 flags.go:64] FLAG: --registry-burst="10" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068228 2563 flags.go:64] FLAG: --registry-qps="5" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068231 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068234 2563 flags.go:64] FLAG: --reserved-memory="" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068238 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068241 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068244 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068247 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068250 2563 flags.go:64] FLAG: --runonce="false" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068253 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 18:40:10.076299 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068256 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068259 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068263 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068266 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068269 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068272 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068275 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068278 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068281 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068284 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068287 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068290 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068294 2563 flags.go:64] FLAG: --system-cgroups="" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068297 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068303 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068306 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068308 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068316 2563 flags.go:64] FLAG: --tls-min-version="" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068319 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068322 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068325 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068328 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068331 2563 flags.go:64] FLAG: --v="2" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068335 2563 flags.go:64] FLAG: --version="false" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068343 2563 flags.go:64] FLAG: --vmodule="" Jul 10 18:40:10.076945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068347 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.068351 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068470 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068476 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068480 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068484 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068487 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068490 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068493 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068496 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068499 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068502 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068505 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068507 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068510 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068513 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068516 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068519 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068522 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068526 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:40:10.077591 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068529 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068532 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068535 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068537 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068541 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068546 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068549 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068552 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068555 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068558 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068561 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068563 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068567 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068571 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068574 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068577 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068579 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068582 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068585 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068588 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:40:10.078132 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068590 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068593 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068596 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068599 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068601 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068604 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068607 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068610 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068612 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068615 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068618 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068622 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068625 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068628 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068630 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068633 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068636 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068639 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068642 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068645 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:40:10.078668 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068647 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068650 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068653 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068655 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068658 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068669 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068672 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068675 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068678 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068681 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068684 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068687 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068690 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068692 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068695 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068698 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068701 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068703 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068706 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068709 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:40:10.079260 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068711 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068714 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068717 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068736 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068739 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068742 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068744 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068747 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068750 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068753 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068755 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068758 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068762 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068765 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068768 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068771 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068774 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068778 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068787 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068791 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:40:10.079792 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.068793 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.069993 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.077185 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.077202 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.077215 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.077225 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077286 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077292 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077295 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077298 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077301 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077304 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077307 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077310 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077313 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:40:10.080337 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077316 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077321 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077325 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077328 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077331 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077334 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077336 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077339 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077342 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077345 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077348 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077352 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077356 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077360 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077363 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077366 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077369 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077372 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077374 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:40:10.080741 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077377 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077381 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077384 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077387 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077390 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077393 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077397 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077399 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077403 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077405 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077408 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077411 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077414 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077417 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077420 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077422 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077425 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077428 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077430 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:40:10.081240 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077433 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077436 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077438 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077441 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077444 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077447 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077449 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077452 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077455 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077458 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077462 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077465 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077467 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077470 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077474 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077477 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077480 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077482 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077485 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077488 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:40:10.081737 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077491 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077493 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077496 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077499 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077501 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077504 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077507 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077509 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077512 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077515 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077517 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077520 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077523 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077526 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077529 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077531 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077534 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077537 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077540 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077543 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:40:10.082257 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077545 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077548 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077551 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077554 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077557 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077561 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077566 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077570 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077573 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077576 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077578 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077581 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.077587 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077704 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077709 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:40:10.082917 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077712 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077716 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077734 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077738 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077741 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077744 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077747 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077750 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077754 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077758 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077762 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077765 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077768 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077770 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077773 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077776 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077779 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077781 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077784 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:40:10.083322 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077787 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077790 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077793 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077795 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077798 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077801 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077804 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077807 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077810 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077812 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077815 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077818 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077821 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077824 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077826 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077830 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077834 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077837 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077840 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:40:10.083806 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077843 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077845 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077848 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077851 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077853 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077856 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077859 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077861 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077864 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077867 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077869 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077872 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077875 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077877 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077880 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077883 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077886 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077889 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077892 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077895 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:40:10.084392 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077898 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077901 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077903 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077906 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077910 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077914 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077917 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077920 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077923 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077926 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077929 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077932 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077934 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077937 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077940 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077943 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077945 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077948 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077951 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077953 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:40:10.084919 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077956 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077959 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077961 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077964 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077967 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077970 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077972 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077975 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077978 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077981 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077984 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077987 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077990 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077992 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077995 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.077998 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.078001 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.078003 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:40:10.085441 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:10.078006 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:40:10.085916 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.078011 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:40:10.085916 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.078104 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 18:40:10.085916 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.080285 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 18:40:10.085916 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.081993 2563 server.go:1019] "Starting client certificate rotation" Jul 10 18:40:10.085916 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.082108 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:40:10.085916 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.082145 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:40:10.085916 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.084802 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:40:10.086115 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.085973 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:40:10.104006 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.103989 2563 log.go:25] "Validated CRI v1 runtime API" Jul 10 18:40:10.106757 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.106743 2563 log.go:25] "Validated CRI v1 image API" Jul 10 18:40:10.107527 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.107509 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 18:40:10.110805 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.110786 2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b5647052-4d8f-4721-9af8-5c6d257de5b9:/dev/nvme0n1p4 ceb118d6-9b21-4535-b715-897e9c537dc3:/dev/nvme0n1p3] Jul 10 18:40:10.110876 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.110804 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 18:40:10.112630 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.112614 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:40:10.116038 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.115915 2563 manager.go:216] Machine: {Timestamp:2026-07-10 18:40:10.114656961 +0000 UTC m=+0.399208284 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3118368 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ed70f93cfed15d543e86c1ee46bb2 SystemUUID:ec2ed70f-93cf-ed15-d543-e86c1ee46bb2 BootID:21734ebb-6a06-4259-8e02-f36912cdb133 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:57:ba:26:10:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:57:ba:26:10:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:03:c1:7e:6e:9a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 18:40:10.116038 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.116032 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 18:40:10.116141 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.116108 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 18:40:10.119292 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.119267 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 18:40:10.119435 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.119294 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-86.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 18:40:10.120120 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.120110 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 18:40:10.120154 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.120122 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 18:40:10.120154 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.120130 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:40:10.121003 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.120992 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:40:10.121865 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.121855 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 18:40:10.125236 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.125225 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:40:10.125356 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.125346 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 18:40:10.125389 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.125377 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 10 18:40:10.125389 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.125387 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 18:40:10.125448 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.125402 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 18:40:10.125448 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.125420 2563 kubelet.go:398] "Adding apiserver pod source" Jul 10 18:40:10.125448 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.125430 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 18:40:10.125793 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.125781 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:40:10.125833 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.125801 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:40:10.128481 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.128463 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 18:40:10.129841 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.129827 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 18:40:10.129884 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.129859 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 18:40:10.130852 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130841 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 18:40:10.130906 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130856 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 18:40:10.130906 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130863 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 18:40:10.130906 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130869 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 18:40:10.130906 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130878 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 18:40:10.130906 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130884 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 18:40:10.130906 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130893 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 18:40:10.130906 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130898 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 18:40:10.130906 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130905 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 18:40:10.131125 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130911 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 18:40:10.131125 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130920 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 18:40:10.131125 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.130926 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 18:40:10.131864 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.131851 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 18:40:10.131919 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.131866 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 18:40:10.135314 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.135298 2563 server.go:1268] "Started kubelet" Jul 10 18:40:10.135437 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.135410 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 18:40:10.135502 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.135400 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 18:40:10.135502 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.135495 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 18:40:10.135906 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.135875 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f9cqn" Jul 10 18:40:10.136144 ip-10-0-137-86 systemd[1]: Started Kubernetes Kubelet. Jul 10 18:40:10.139341 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.139309 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 18:40:10.139714 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.139685 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-86.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 18:40:10.139825 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.139786 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 18:40:10.140035 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.140010 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-86.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 18:40:10.141540 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.141513 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 10 18:40:10.143183 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.143160 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 18:40:10.143915 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.143898 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 18:40:10.144007 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.143914 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 18:40:10.144007 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.143966 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:10.144466 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.144447 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f9cqn" Jul 10 18:40:10.144724 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.144558 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 18:40:10.144724 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.144580 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 18:40:10.144724 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.144557 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 18:40:10.144914 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.144756 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 10 18:40:10.144914 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.144770 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 10 18:40:10.145312 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.145202 2563 factory.go:55] Registering systemd factory Jul 10 18:40:10.145312 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.145220 2563 factory.go:223] Registration of the systemd container factory successfully Jul 10 18:40:10.145638 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.145505 2563 factory.go:153] Registering CRI-O factory Jul 10 18:40:10.145638 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.145521 2563 factory.go:223] Registration of the crio container factory successfully Jul 10 18:40:10.145638 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.145576 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 18:40:10.145638 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.145603 2563 factory.go:103] Registering Raw factory Jul 10 18:40:10.145638 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.145621 2563 manager.go:1196] Started watching for new ooms in manager Jul 10 18:40:10.146607 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.146589 2563 manager.go:349] Starting recovery of all containers Jul 10 18:40:10.146741 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.145571 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-86.ec2.internal.18c101b65ad8f668 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-86.ec2.internal,UID:ip-10-0-137-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-86.ec2.internal,},FirstTimestamp:2026-07-10 18:40:10.135271016 +0000 UTC m=+0.419822316,LastTimestamp:2026-07-10 18:40:10.135271016 +0000 UTC m=+0.419822316,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-86.ec2.internal,}" Jul 10 18:40:10.154899 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.154881 2563 manager.go:354] Recovery completed Jul 10 18:40:10.157854 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.157824 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:40:10.160088 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.160074 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:40:10.160642 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.160625 2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-86.ec2.internal\" not found" node="ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.161958 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.161944 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:40:10.162019 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.161970 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:40:10.162019 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.161981 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:40:10.162489 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.162473 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 18:40:10.162552 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.162490 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 18:40:10.162552 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.162512 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:40:10.162552 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.162522 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:40:10.163311 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.163298 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:40:10.163367 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.163324 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:40:10.163367 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.163338 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:40:10.163811 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.163801 2563 policy_none.go:49] "None policy: Start" Jul 10 18:40:10.163852 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.163815 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 18:40:10.163852 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.163826 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 18:40:10.164855 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.164841 2563 policy_none.go:47] "Start" Jul 10 18:40:10.206434 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.206266 2563 manager.go:337] "Starting Device Plugin manager" Jul 10 18:40:10.206506 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.206450 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 18:40:10.206506 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.206462 2563 server.go:87] "Starting device plugin registration server" Jul 10 18:40:10.206658 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.206646 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 18:40:10.206658 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.206659 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 18:40:10.206847 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.206822 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 18:40:10.206944 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.206929 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 18:40:10.207005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.206946 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 18:40:10.207409 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.207384 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 18:40:10.207494 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.207444 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:10.307037 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.306971 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:40:10.307983 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.307904 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:40:10.307983 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.307930 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:40:10.307983 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.307941 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:40:10.307983 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.307967 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.315541 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.315528 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.315643 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.315546 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-86.ec2.internal\": node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:10.332352 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.332321 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 18:40:10.332425 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.332355 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:10.333591 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.333576 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 18:40:10.333649 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.333612 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 18:40:10.333649 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.333631 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 18:40:10.333739 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.333664 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 18:40:10.335993 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.335972 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:40:10.436395 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.436350 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:10.436395 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.436367 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-86.ec2.internal"] Jul 10 18:40:10.436567 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.436442 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:40:10.438109 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.438087 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:40:10.438208 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.438120 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:40:10.438208 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.438132 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:40:10.439535 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.439523 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:40:10.439635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.439621 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.439708 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.439694 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:40:10.440362 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.440334 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:40:10.440446 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.440378 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:40:10.440446 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.440391 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:40:10.440446 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.440337 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:40:10.440601 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.440470 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:40:10.440601 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.440490 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:40:10.441515 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.441501 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.441590 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.441526 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:40:10.442328 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.442309 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:40:10.442421 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.442339 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:40:10.442421 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.442364 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:40:10.446060 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.446042 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e830fa69557184eff7ed9c13b4c4be24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal\" (UID: \"e830fa69557184eff7ed9c13b4c4be24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.446150 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.446077 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e830fa69557184eff7ed9c13b4c4be24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal\" (UID: \"e830fa69557184eff7ed9c13b4c4be24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.446150 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.446102 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f43f038d238bfb0aa554060566ad6e31-config\") pod \"kube-apiserver-proxy-ip-10-0-137-86.ec2.internal\" (UID: \"f43f038d238bfb0aa554060566ad6e31\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.466367 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.466345 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-86.ec2.internal\" not found" node="ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.471121 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.471103 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-86.ec2.internal\" not found" node="ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.537280 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.537259 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:10.546222 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.546201 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f43f038d238bfb0aa554060566ad6e31-config\") pod \"kube-apiserver-proxy-ip-10-0-137-86.ec2.internal\" (UID: \"f43f038d238bfb0aa554060566ad6e31\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.546281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.546232 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e830fa69557184eff7ed9c13b4c4be24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal\" (UID: \"e830fa69557184eff7ed9c13b4c4be24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.546281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.546250 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e830fa69557184eff7ed9c13b4c4be24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal\" (UID: \"e830fa69557184eff7ed9c13b4c4be24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.546398 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.546293 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e830fa69557184eff7ed9c13b4c4be24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal\" (UID: \"e830fa69557184eff7ed9c13b4c4be24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.546398 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.546327 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f43f038d238bfb0aa554060566ad6e31-config\") pod \"kube-apiserver-proxy-ip-10-0-137-86.ec2.internal\" (UID: \"f43f038d238bfb0aa554060566ad6e31\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.546398 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.546328 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e830fa69557184eff7ed9c13b4c4be24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal\" (UID: \"e830fa69557184eff7ed9c13b4c4be24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.637533 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.637462 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:10.739064 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.739026 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:10.769793 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.769769 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.775105 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:10.774299 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-86.ec2.internal" Jul 10 18:40:10.841265 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.841234 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:10.949025 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:10.948937 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:11.049923 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:11.049891 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:11.082534 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.082498 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 18:40:11.082991 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.082683 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 18:40:11.082991 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.082718 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 18:40:11.149286 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.149248 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 18:35:10 +0000 UTC" deadline="2028-02-06 13:11:24.491026838 +0000 UTC" Jul 10 18:40:11.149286 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.149278 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13818h31m13.341752334s" Jul 10 18:40:11.150441 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:11.150422 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:11.250553 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:11.250479 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:11.254418 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.254396 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:40:11.351190 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:11.351167 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:11.452269 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:11.452249 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:11.552967 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:11.552906 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-86.ec2.internal\" not found" Jul 10 18:40:11.566672 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.566652 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:40:11.574167 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:11.574139 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf43f038d238bfb0aa554060566ad6e31.slice/crio-102eb0979982763e4452fed22f00bbe299e9299c2378c85689abd03b2d2a9aec WatchSource:0}: Error finding container 102eb0979982763e4452fed22f00bbe299e9299c2378c85689abd03b2d2a9aec: Status 404 returned error can't find the container with id 102eb0979982763e4452fed22f00bbe299e9299c2378c85689abd03b2d2a9aec Jul 10 18:40:11.574582 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:11.574558 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode830fa69557184eff7ed9c13b4c4be24.slice/crio-e63309ddee7427634fbfd1c4721355419a268ebbcae8aac1fc24a01107449c64 WatchSource:0}: Error finding container e63309ddee7427634fbfd1c4721355419a268ebbcae8aac1fc24a01107449c64: Status 404 returned error can't find the container with id e63309ddee7427634fbfd1c4721355419a268ebbcae8aac1fc24a01107449c64 Jul 10 18:40:11.579152 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.579137 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:40:11.644942 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.644912 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" Jul 10 18:40:11.661096 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.661075 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:40:11.662295 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.662282 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-86.ec2.internal" Jul 10 18:40:11.669452 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:11.669436 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:40:12.127945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.127910 2563 apiserver.go:52] "Watching apiserver" Jul 10 18:40:12.138639 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.138608 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 18:40:12.139668 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.139644 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-xksn6","openshift-multus/multus-additional-cni-plugins-xgvw7","openshift-multus/network-metrics-daemon-ccjtn","openshift-network-operator/iptables-alerter-8bbcs","kube-system/kube-apiserver-proxy-ip-10-0-137-86.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw","openshift-cluster-node-tuning-operator/tuned-jwrjq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal","openshift-multus/multus-brxbd","openshift-network-diagnostics/network-check-target-prjp9","openshift-ovn-kubernetes/ovnkube-node-gf64p","kube-system/global-pull-secret-syncer-dj7wf","kube-system/konnectivity-agent-ltklt","openshift-dns/node-resolver-4xz7g"] Jul 10 18:40:12.141393 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.141358 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:12.141494 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.141442 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:12.142713 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.142632 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.144060 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.144033 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.145141 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.144920 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 18:40:12.145141 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.145018 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b56pt\"" Jul 10 18:40:12.145141 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.145084 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 18:40:12.145141 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.145134 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 18:40:12.145428 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.145409 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 18:40:12.145486 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.145454 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 18:40:12.145671 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.145653 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 18:40:12.145782 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.145684 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8bbcs" Jul 10 18:40:12.146293 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.146266 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:40:12.146293 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.146276 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 18:40:12.146870 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.146829 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nnp7c\"" Jul 10 18:40:12.146979 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.146962 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.148147 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.148123 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 18:40:12.148246 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.148164 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:40:12.148246 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.148171 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 18:40:12.148246 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.148202 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gkqqh\"" Jul 10 18:40:12.148595 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.148576 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xksn6" Jul 10 18:40:12.149543 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.149526 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 18:40:12.149639 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.149609 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 18:40:12.149639 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.149620 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zfwfb\"" Jul 10 18:40:12.149962 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.149946 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.150069 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.149989 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 18:40:12.150847 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.150825 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 18:40:12.150942 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.150928 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 18:40:12.151436 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.151211 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bd2qc\"" Jul 10 18:40:12.151436 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.151216 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 18:40:12.151867 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.151669 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:12.151867 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.151726 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:12.152330 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.152300 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 18:40:12.152419 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.152355 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9hb7r\"" Jul 10 18:40:12.153457 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.153438 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.155109 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.155089 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:12.155208 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.155156 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:12.155382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.155361 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:40:12.155760 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.155741 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 18:40:12.155879 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.155838 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 18:40:12.155879 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.155814 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db8rp\" (UniqueName: \"kubernetes.io/projected/b22d6861-0651-43b0-8642-f1bb3a2c54ee-kube-api-access-db8rp\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:12.156029 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.155879 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7788073c-5d44-49cd-9ae6-51a11ac16848-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.156029 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.155892 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 18:40:12.156029 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.155900 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c8mhw\"" Jul 10 18:40:12.156029 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.155915 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-modprobe-d\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.156029 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.155941 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-var-lib-kubelet\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.156029 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.155995 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-system-cni-dir\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.156382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156032 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-cnibin\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.156382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156058 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7788073c-5d44-49cd-9ae6-51a11ac16848-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.156382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156119 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c31d3759-3875-4826-bf22-eafe856a5172-etc-tuned\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.156382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156167 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrkvc\" (UniqueName: \"kubernetes.io/projected/c31d3759-3875-4826-bf22-eafe856a5172-kube-api-access-zrkvc\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.156382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156205 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 18:40:12.156382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156227 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/944afe1e-fa5a-4b53-903e-25329fec3529-host-slash\") pod \"iptables-alerter-8bbcs\" (UID: \"944afe1e-fa5a-4b53-903e-25329fec3529\") " pod="openshift-network-operator/iptables-alerter-8bbcs" Jul 10 18:40:12.156382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156258 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7788073c-5d44-49cd-9ae6-51a11ac16848-cni-binary-copy\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.156382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156299 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpkf6\" (UniqueName: \"kubernetes.io/projected/944afe1e-fa5a-4b53-903e-25329fec3529-kube-api-access-qpkf6\") pod \"iptables-alerter-8bbcs\" (UID: \"944afe1e-fa5a-4b53-903e-25329fec3529\") " pod="openshift-network-operator/iptables-alerter-8bbcs" Jul 10 18:40:12.156382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156348 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-lib-modules\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.156382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156379 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/944afe1e-fa5a-4b53-903e-25329fec3529-iptables-alerter-script\") pod \"iptables-alerter-8bbcs\" (UID: \"944afe1e-fa5a-4b53-903e-25329fec3529\") " pod="openshift-network-operator/iptables-alerter-8bbcs" Jul 10 18:40:12.157005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156406 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-sysconfig\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.157005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156429 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-sysctl-conf\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.157005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156454 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-systemd\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.157005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:12.157005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156504 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-os-release\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.157005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156524 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 18:40:12.157005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156610 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6lrp\" (UniqueName: \"kubernetes.io/projected/7788073c-5d44-49cd-9ae6-51a11ac16848-kube-api-access-p6lrp\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.157005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156654 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-kubernetes\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.157005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.156771 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:12.157469 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.157047 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 18:40:12.157469 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.157158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-run\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.157469 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.157192 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-sys\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.157469 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.157228 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-host\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.157469 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.157265 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c31d3759-3875-4826-bf22-eafe856a5172-tmp\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.157469 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.157300 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.157469 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.157330 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-sysctl-d\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.160441 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.160419 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4xz7g" Jul 10 18:40:12.164098 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.164076 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 18:40:12.164350 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.164331 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-z9qh7\"" Jul 10 18:40:12.164473 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.164456 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 18:40:12.165553 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.165530 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 18:40:12.165711 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.165692 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 18:40:12.166044 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.166025 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xvwvh\"" Jul 10 18:40:12.186110 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.186072 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dfwv8" Jul 10 18:40:12.197290 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.196880 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dfwv8" Jul 10 18:40:12.246567 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.246546 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 18:40:12.257919 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.257840 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7788073c-5d44-49cd-9ae6-51a11ac16848-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.257919 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.257883 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrkvc\" (UniqueName: \"kubernetes.io/projected/c31d3759-3875-4826-bf22-eafe856a5172-kube-api-access-zrkvc\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.257919 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.257913 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/988b88a2-e62e-4676-91e4-01157f069202-serviceca\") pod \"node-ca-xksn6\" (UID: \"988b88a2-e62e-4676-91e4-01157f069202\") " pod="openshift-image-registry/node-ca-xksn6" Jul 10 18:40:12.258164 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.257969 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-var-lib-kubelet\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.258305 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258288 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-run-multus-certs\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.258374 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258341 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpkf6\" (UniqueName: \"kubernetes.io/projected/944afe1e-fa5a-4b53-903e-25329fec3529-kube-api-access-qpkf6\") pod \"iptables-alerter-8bbcs\" (UID: \"944afe1e-fa5a-4b53-903e-25329fec3529\") " pod="openshift-network-operator/iptables-alerter-8bbcs" Jul 10 18:40:12.258426 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258373 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.258426 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258401 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-cni-netd\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.258537 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258443 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b65b77e-09b9-4937-8014-6431509be7b6-konnectivity-ca\") pod \"konnectivity-agent-ltklt\" (UID: \"4b65b77e-09b9-4937-8014-6431509be7b6\") " pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:12.258537 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258474 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.258537 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258502 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/364b43bc-cd7a-49cb-b214-cdbece4997df-env-overrides\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.258689 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258542 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-sysconfig\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.258689 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258571 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-sysctl-conf\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.258689 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258596 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-sys-fs\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.258689 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258619 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-hostroot\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.259106 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258948 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-os-release\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.259106 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.258985 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-etc-kubernetes\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.259106 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259048 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-sysconfig\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.259106 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259098 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr2mx\" (UniqueName: \"kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx\") pod \"network-check-target-prjp9\" (UID: \"b691ab8e-52aa-40ea-9e2b-ce8d242181ef\") " pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259131 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259157 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-sysctl-conf\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259161 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-kubernetes\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259196 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-run\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259210 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7788073c-5d44-49cd-9ae6-51a11ac16848-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259230 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-kubernetes\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259236 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-os-release\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259259 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-sys\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259282 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-host\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259314 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-socket-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259333 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-etc-selinux\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259344 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-sys\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259363 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtgkw\" (UniqueName: \"kubernetes.io/projected/988b88a2-e62e-4676-91e4-01157f069202-kube-api-access-dtgkw\") pod \"node-ca-xksn6\" (UID: \"988b88a2-e62e-4676-91e4-01157f069202\") " pod="openshift-image-registry/node-ca-xksn6" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259365 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-run\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259381 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-os-release\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259395 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-host\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.259392 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259405 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259448 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-sysctl-d\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a16c8e1-488a-4082-a279-724ea112e68f-cni-binary-copy\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259506 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-systemd-units\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259552 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-run-netns\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259578 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-node-log\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259619 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7788073c-5d44-49cd-9ae6-51a11ac16848-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259620 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-sysctl-d\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259622 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259657 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-modprobe-d\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259719 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-var-lib-kubelet\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259797 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-multus-cni-dir\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259829 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-etc-openvswitch\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259858 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-cni-bin\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259904 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c31d3759-3875-4826-bf22-eafe856a5172-etc-tuned\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259931 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/944afe1e-fa5a-4b53-903e-25329fec3529-host-slash\") pod \"iptables-alerter-8bbcs\" (UID: \"944afe1e-fa5a-4b53-903e-25329fec3529\") " pod="openshift-network-operator/iptables-alerter-8bbcs" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259958 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-var-lib-cni-bin\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.260380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.259986 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-slash\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260013 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/364b43bc-cd7a-49cb-b214-cdbece4997df-ovnkube-config\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260040 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-kubelet-config\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260131 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrphd\" (UniqueName: \"kubernetes.io/projected/cb1e6736-8a0c-4dcd-bef2-a88218277f25-kube-api-access-rrphd\") pod \"node-resolver-4xz7g\" (UID: \"cb1e6736-8a0c-4dcd-bef2-a88218277f25\") " pod="openshift-dns/node-resolver-4xz7g" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260162 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7788073c-5d44-49cd-9ae6-51a11ac16848-cni-binary-copy\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260187 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-system-cni-dir\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260210 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7788073c-5d44-49cd-9ae6-51a11ac16848-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260216 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-run-ovn-kubernetes\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260260 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-lib-modules\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260291 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/944afe1e-fa5a-4b53-903e-25329fec3529-iptables-alerter-script\") pod \"iptables-alerter-8bbcs\" (UID: \"944afe1e-fa5a-4b53-903e-25329fec3529\") " pod="openshift-network-operator/iptables-alerter-8bbcs" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260313 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-modprobe-d\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-cnibin\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260369 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-var-lib-kubelet\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260482 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-lib-modules\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260589 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.260794 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/944afe1e-fa5a-4b53-903e-25329fec3529-host-slash\") pod \"iptables-alerter-8bbcs\" (UID: \"944afe1e-fa5a-4b53-903e-25329fec3529\") " pod="openshift-network-operator/iptables-alerter-8bbcs" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261005 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-var-lib-cni-multus\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.261259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261012 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/944afe1e-fa5a-4b53-903e-25329fec3529-iptables-alerter-script\") pod \"iptables-alerter-8bbcs\" (UID: \"944afe1e-fa5a-4b53-903e-25329fec3529\") " pod="openshift-network-operator/iptables-alerter-8bbcs" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261044 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-multus-conf-dir\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261101 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbjr4\" (UniqueName: \"kubernetes.io/projected/4a16c8e1-488a-4082-a279-724ea112e68f-kube-api-access-cbjr4\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261159 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-kubelet\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-run-systemd\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261231 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-systemd\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261266 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-run-openvswitch\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261295 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-run-ovn\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261316 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c31d3759-3875-4826-bf22-eafe856a5172-etc-systemd\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261321 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/364b43bc-cd7a-49cb-b214-cdbece4997df-ovnkube-script-lib\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261347 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cdgx\" (UniqueName: \"kubernetes.io/projected/364b43bc-cd7a-49cb-b214-cdbece4997df-kube-api-access-8cdgx\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261333 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7788073c-5d44-49cd-9ae6-51a11ac16848-cni-binary-copy\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261381 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-dbus\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261420 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b65b77e-09b9-4937-8014-6431509be7b6-agent-certs\") pod \"konnectivity-agent-ltklt\" (UID: \"4b65b77e-09b9-4937-8014-6431509be7b6\") " pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261453 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.261543 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:12.262119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261531 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc99k\" (UniqueName: \"kubernetes.io/projected/3f00c215-ee46-4329-b991-55e2d1a4912f-kube-api-access-fc99k\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.261652 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs podName:b22d6861-0651-43b0-8642-f1bb3a2c54ee nodeName:}" failed. No retries permitted until 2026-07-10 18:40:12.761630736 +0000 UTC m=+3.046182029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs") pod "network-metrics-daemon-ccjtn" (UID: "b22d6861-0651-43b0-8642-f1bb3a2c54ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261683 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a16c8e1-488a-4082-a279-724ea112e68f-multus-daemon-config\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261739 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6lrp\" (UniqueName: \"kubernetes.io/projected/7788073c-5d44-49cd-9ae6-51a11ac16848-kube-api-access-p6lrp\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261822 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c31d3759-3875-4826-bf22-eafe856a5172-tmp\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261864 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-multus-socket-dir-parent\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261909 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb1e6736-8a0c-4dcd-bef2-a88218277f25-hosts-file\") pod \"node-resolver-4xz7g\" (UID: \"cb1e6736-8a0c-4dcd-bef2-a88218277f25\") " pod="openshift-dns/node-resolver-4xz7g" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261936 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-device-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261962 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-run-k8s-cni-cncf-io\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.261987 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-log-socket\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.262009 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb1e6736-8a0c-4dcd-bef2-a88218277f25-tmp-dir\") pod \"node-resolver-4xz7g\" (UID: \"cb1e6736-8a0c-4dcd-bef2-a88218277f25\") " pod="openshift-dns/node-resolver-4xz7g" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.262038 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-run-netns\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.262068 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db8rp\" (UniqueName: \"kubernetes.io/projected/b22d6861-0651-43b0-8642-f1bb3a2c54ee-kube-api-access-db8rp\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.262106 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-registration-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.262137 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/988b88a2-e62e-4676-91e4-01157f069202-host\") pod \"node-ca-xksn6\" (UID: \"988b88a2-e62e-4676-91e4-01157f069202\") " pod="openshift-image-registry/node-ca-xksn6" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.262161 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-var-lib-openvswitch\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.263057 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.262209 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/364b43bc-cd7a-49cb-b214-cdbece4997df-ovn-node-metrics-cert\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.263962 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.262239 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-system-cni-dir\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.263962 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.262266 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-cnibin\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.263962 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.262368 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-cnibin\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.263962 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.262416 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7788073c-5d44-49cd-9ae6-51a11ac16848-system-cni-dir\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.267381 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.267322 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c31d3759-3875-4826-bf22-eafe856a5172-etc-tuned\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.268078 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.268038 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c31d3759-3875-4826-bf22-eafe856a5172-tmp\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.268801 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.268775 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrkvc\" (UniqueName: \"kubernetes.io/projected/c31d3759-3875-4826-bf22-eafe856a5172-kube-api-access-zrkvc\") pod \"tuned-jwrjq\" (UID: \"c31d3759-3875-4826-bf22-eafe856a5172\") " pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.273533 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.273506 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6lrp\" (UniqueName: \"kubernetes.io/projected/7788073c-5d44-49cd-9ae6-51a11ac16848-kube-api-access-p6lrp\") pod \"multus-additional-cni-plugins-xgvw7\" (UID: \"7788073c-5d44-49cd-9ae6-51a11ac16848\") " pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.274387 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.274342 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpkf6\" (UniqueName: \"kubernetes.io/projected/944afe1e-fa5a-4b53-903e-25329fec3529-kube-api-access-qpkf6\") pod \"iptables-alerter-8bbcs\" (UID: \"944afe1e-fa5a-4b53-903e-25329fec3529\") " pod="openshift-network-operator/iptables-alerter-8bbcs" Jul 10 18:40:12.276256 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.276232 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db8rp\" (UniqueName: \"kubernetes.io/projected/b22d6861-0651-43b0-8642-f1bb3a2c54ee-kube-api-access-db8rp\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:12.339508 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.339453 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" event={"ID":"e830fa69557184eff7ed9c13b4c4be24","Type":"ContainerStarted","Data":"e63309ddee7427634fbfd1c4721355419a268ebbcae8aac1fc24a01107449c64"} Jul 10 18:40:12.341886 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.341858 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-86.ec2.internal" event={"ID":"f43f038d238bfb0aa554060566ad6e31","Type":"ContainerStarted","Data":"102eb0979982763e4452fed22f00bbe299e9299c2378c85689abd03b2d2a9aec"} Jul 10 18:40:12.363385 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363304 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-device-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.363385 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-run-k8s-cni-cncf-io\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.363385 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363373 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-log-socket\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363403 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb1e6736-8a0c-4dcd-bef2-a88218277f25-tmp-dir\") pod \"node-resolver-4xz7g\" (UID: \"cb1e6736-8a0c-4dcd-bef2-a88218277f25\") " pod="openshift-dns/node-resolver-4xz7g" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363428 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-run-netns\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363435 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-device-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363456 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-registration-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/988b88a2-e62e-4676-91e4-01157f069202-host\") pod \"node-ca-xksn6\" (UID: \"988b88a2-e62e-4676-91e4-01157f069202\") " pod="openshift-image-registry/node-ca-xksn6" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363515 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-var-lib-openvswitch\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363529 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-log-socket\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/364b43bc-cd7a-49cb-b214-cdbece4997df-ovn-node-metrics-cert\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363569 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/988b88a2-e62e-4676-91e4-01157f069202-serviceca\") pod \"node-ca-xksn6\" (UID: \"988b88a2-e62e-4676-91e4-01157f069202\") " pod="openshift-image-registry/node-ca-xksn6" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363589 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-run-k8s-cni-cncf-io\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363593 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-var-lib-kubelet\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.363635 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363632 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/988b88a2-e62e-4676-91e4-01157f069202-host\") pod \"node-ca-xksn6\" (UID: \"988b88a2-e62e-4676-91e4-01157f069202\") " pod="openshift-image-registry/node-ca-xksn6" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363654 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-var-lib-kubelet\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363699 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-run-multus-certs\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363722 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-var-lib-openvswitch\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363731 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363758 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-cni-netd\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363784 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b65b77e-09b9-4937-8014-6431509be7b6-konnectivity-ca\") pod \"konnectivity-agent-ltklt\" (UID: \"4b65b77e-09b9-4937-8014-6431509be7b6\") " pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363827 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363857 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/364b43bc-cd7a-49cb-b214-cdbece4997df-env-overrides\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363887 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-sys-fs\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363911 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-hostroot\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363947 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-etc-kubernetes\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.363974 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr2mx\" (UniqueName: \"kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx\") pod \"network-check-target-prjp9\" (UID: \"b691ab8e-52aa-40ea-9e2b-ce8d242181ef\") " pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364030 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-socket-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364054 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-etc-selinux\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364085 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtgkw\" (UniqueName: \"kubernetes.io/projected/988b88a2-e62e-4676-91e4-01157f069202-kube-api-access-dtgkw\") pod \"node-ca-xksn6\" (UID: \"988b88a2-e62e-4676-91e4-01157f069202\") " pod="openshift-image-registry/node-ca-xksn6" Jul 10 18:40:12.364342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364110 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-os-release\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364136 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a16c8e1-488a-4082-a279-724ea112e68f-cni-binary-copy\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364164 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-systemd-units\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364188 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-run-netns\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364219 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-node-log\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364256 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-multus-cni-dir\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364282 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-etc-openvswitch\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-cni-bin\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364333 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-var-lib-cni-bin\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364359 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-slash\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364385 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/364b43bc-cd7a-49cb-b214-cdbece4997df-ovnkube-config\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364412 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-kubelet-config\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364437 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrphd\" (UniqueName: \"kubernetes.io/projected/cb1e6736-8a0c-4dcd-bef2-a88218277f25-kube-api-access-rrphd\") pod \"node-resolver-4xz7g\" (UID: \"cb1e6736-8a0c-4dcd-bef2-a88218277f25\") " pod="openshift-dns/node-resolver-4xz7g" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364463 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-system-cni-dir\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364491 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-run-ovn-kubernetes\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364519 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-cnibin\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364543 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-var-lib-cni-multus\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.364568 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-multus-conf-dir\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.365200 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.365036 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb1e6736-8a0c-4dcd-bef2-a88218277f25-tmp-dir\") pod \"node-resolver-4xz7g\" (UID: \"cb1e6736-8a0c-4dcd-bef2-a88218277f25\") " pod="openshift-dns/node-resolver-4xz7g" Jul 10 18:40:12.366170 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.365098 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-run-netns\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.366170 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.365148 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-registration-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.366170 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.365184 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-run-multus-certs\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.366170 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.365234 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.366170 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.365627 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-var-lib-cni-bin\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.366170 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.365677 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-run-netns\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.366170 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.365685 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-node-log\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.366170 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.365776 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-cni-netd\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.366170 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.365827 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:12.366170 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.365889 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret podName:ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:12.865868301 +0000 UTC m=+3.150419603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret") pod "global-pull-secret-syncer-dj7wf" (UID: "ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:12.366170 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.366092 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-etc-kubernetes\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.366317 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b65b77e-09b9-4937-8014-6431509be7b6-konnectivity-ca\") pod \"konnectivity-agent-ltklt\" (UID: \"4b65b77e-09b9-4937-8014-6431509be7b6\") " pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.366369 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-systemd-units\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.366903 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a16c8e1-488a-4082-a279-724ea112e68f-cni-binary-copy\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.366945 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-sys-fs\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.366963 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-slash\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367011 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367052 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-hostroot\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367095 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-etc-openvswitch\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367153 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-os-release\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.366320 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-cni-bin\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367169 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/364b43bc-cd7a-49cb-b214-cdbece4997df-ovnkube-config\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367219 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-kubelet-config\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367311 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-multus-cni-dir\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367314 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-run-ovn-kubernetes\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367363 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/364b43bc-cd7a-49cb-b214-cdbece4997df-env-overrides\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367372 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-system-cni-dir\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367493 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-etc-selinux\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.367640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367486 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3f00c215-ee46-4329-b991-55e2d1a4912f-socket-dir\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367541 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-cnibin\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367543 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-host-var-lib-cni-multus\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367626 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbjr4\" (UniqueName: \"kubernetes.io/projected/4a16c8e1-488a-4082-a279-724ea112e68f-kube-api-access-cbjr4\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367684 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-kubelet\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367697 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/988b88a2-e62e-4676-91e4-01157f069202-serviceca\") pod \"node-ca-xksn6\" (UID: \"988b88a2-e62e-4676-91e4-01157f069202\") " pod="openshift-image-registry/node-ca-xksn6" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367710 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-run-systemd\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367750 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-run-openvswitch\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367774 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-run-ovn\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/364b43bc-cd7a-49cb-b214-cdbece4997df-ovnkube-script-lib\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cdgx\" (UniqueName: \"kubernetes.io/projected/364b43bc-cd7a-49cb-b214-cdbece4997df-kube-api-access-8cdgx\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367849 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-dbus\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367873 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b65b77e-09b9-4937-8014-6431509be7b6-agent-certs\") pod \"konnectivity-agent-ltklt\" (UID: \"4b65b77e-09b9-4937-8014-6431509be7b6\") " pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367910 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc99k\" (UniqueName: \"kubernetes.io/projected/3f00c215-ee46-4329-b991-55e2d1a4912f-kube-api-access-fc99k\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367936 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a16c8e1-488a-4082-a279-724ea112e68f-multus-daemon-config\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367966 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-multus-socket-dir-parent\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.368011 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb1e6736-8a0c-4dcd-bef2-a88218277f25-hosts-file\") pod \"node-resolver-4xz7g\" (UID: \"cb1e6736-8a0c-4dcd-bef2-a88218277f25\") " pod="openshift-dns/node-resolver-4xz7g" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.368094 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb1e6736-8a0c-4dcd-bef2-a88218277f25-hosts-file\") pod \"node-resolver-4xz7g\" (UID: \"cb1e6736-8a0c-4dcd-bef2-a88218277f25\") " pod="openshift-dns/node-resolver-4xz7g" Jul 10 18:40:12.368592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.368137 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-host-kubelet\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.369797 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.368181 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-run-systemd\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.369797 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.368226 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-run-openvswitch\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.369797 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.368265 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/364b43bc-cd7a-49cb-b214-cdbece4997df-run-ovn\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.369797 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.368813 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/364b43bc-cd7a-49cb-b214-cdbece4997df-ovnkube-script-lib\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.369797 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.369245 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-dbus\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:12.369797 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.369285 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a16c8e1-488a-4082-a279-724ea112e68f-multus-daemon-config\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.369797 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.367937 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-multus-conf-dir\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.369797 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.369373 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a16c8e1-488a-4082-a279-724ea112e68f-multus-socket-dir-parent\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.370915 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.370847 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/364b43bc-cd7a-49cb-b214-cdbece4997df-ovn-node-metrics-cert\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.371950 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.371925 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b65b77e-09b9-4937-8014-6431509be7b6-agent-certs\") pod \"konnectivity-agent-ltklt\" (UID: \"4b65b77e-09b9-4937-8014-6431509be7b6\") " pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:12.375308 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.375287 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:40:12.375418 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.375317 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:40:12.375418 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.375331 2563 projected.go:196] Error preparing data for projected volume kube-api-access-gr2mx for pod openshift-network-diagnostics/network-check-target-prjp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:12.375533 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.375434 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx podName:b691ab8e-52aa-40ea-9e2b-ce8d242181ef nodeName:}" failed. No retries permitted until 2026-07-10 18:40:12.875418173 +0000 UTC m=+3.159969482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gr2mx" (UniqueName: "kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx") pod "network-check-target-prjp9" (UID: "b691ab8e-52aa-40ea-9e2b-ce8d242181ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:12.378254 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.378191 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrphd\" (UniqueName: \"kubernetes.io/projected/cb1e6736-8a0c-4dcd-bef2-a88218277f25-kube-api-access-rrphd\") pod \"node-resolver-4xz7g\" (UID: \"cb1e6736-8a0c-4dcd-bef2-a88218277f25\") " pod="openshift-dns/node-resolver-4xz7g" Jul 10 18:40:12.378979 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.378951 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtgkw\" (UniqueName: \"kubernetes.io/projected/988b88a2-e62e-4676-91e4-01157f069202-kube-api-access-dtgkw\") pod \"node-ca-xksn6\" (UID: \"988b88a2-e62e-4676-91e4-01157f069202\") " pod="openshift-image-registry/node-ca-xksn6" Jul 10 18:40:12.380032 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.380003 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cdgx\" (UniqueName: \"kubernetes.io/projected/364b43bc-cd7a-49cb-b214-cdbece4997df-kube-api-access-8cdgx\") pod \"ovnkube-node-gf64p\" (UID: \"364b43bc-cd7a-49cb-b214-cdbece4997df\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.380650 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.380623 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbjr4\" (UniqueName: \"kubernetes.io/projected/4a16c8e1-488a-4082-a279-724ea112e68f-kube-api-access-cbjr4\") pod \"multus-brxbd\" (UID: \"4a16c8e1-488a-4082-a279-724ea112e68f\") " pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.380898 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.380826 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc99k\" (UniqueName: \"kubernetes.io/projected/3f00c215-ee46-4329-b991-55e2d1a4912f-kube-api-access-fc99k\") pod \"aws-ebs-csi-driver-node-p66bw\" (UID: \"3f00c215-ee46-4329-b991-55e2d1a4912f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.456296 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.456262 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xgvw7" Jul 10 18:40:12.464657 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.464626 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" Jul 10 18:40:12.466121 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:12.466094 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7788073c_5d44_49cd_9ae6_51a11ac16848.slice/crio-84cd68d2b4d5b98ea3777db7a34ab3ace92023f29097f86653098abf8b7d24da WatchSource:0}: Error finding container 84cd68d2b4d5b98ea3777db7a34ab3ace92023f29097f86653098abf8b7d24da: Status 404 returned error can't find the container with id 84cd68d2b4d5b98ea3777db7a34ab3ace92023f29097f86653098abf8b7d24da Jul 10 18:40:12.473640 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:12.473615 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc31d3759_3875_4826_bf22_eafe856a5172.slice/crio-0110e93b00497000d6c91139cff7abba876d748152112cc60ed03b9f698cd9d1 WatchSource:0}: Error finding container 0110e93b00497000d6c91139cff7abba876d748152112cc60ed03b9f698cd9d1: Status 404 returned error can't find the container with id 0110e93b00497000d6c91139cff7abba876d748152112cc60ed03b9f698cd9d1 Jul 10 18:40:12.476383 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.475756 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8bbcs" Jul 10 18:40:12.483433 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.483413 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" Jul 10 18:40:12.486175 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:12.486145 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod944afe1e_fa5a_4b53_903e_25329fec3529.slice/crio-0547e0697f6bef3ff55353392bfc938b825d129922f381fc64865158456b4ce0 WatchSource:0}: Error finding container 0547e0697f6bef3ff55353392bfc938b825d129922f381fc64865158456b4ce0: Status 404 returned error can't find the container with id 0547e0697f6bef3ff55353392bfc938b825d129922f381fc64865158456b4ce0 Jul 10 18:40:12.492210 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.492051 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xksn6" Jul 10 18:40:12.494859 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:12.494829 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f00c215_ee46_4329_b991_55e2d1a4912f.slice/crio-80b62478c82d34f06e0ac90c0d7d633ab89039a637a62e0adfc7395c947b62d8 WatchSource:0}: Error finding container 80b62478c82d34f06e0ac90c0d7d633ab89039a637a62e0adfc7395c947b62d8: Status 404 returned error can't find the container with id 80b62478c82d34f06e0ac90c0d7d633ab89039a637a62e0adfc7395c947b62d8 Jul 10 18:40:12.501084 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.500925 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-brxbd" Jul 10 18:40:12.502653 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:12.502628 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod988b88a2_e62e_4676_91e4_01157f069202.slice/crio-e332310e72ea67ef5664fd9b52570ce4331381a7431bf92b4bd50d52525694fc WatchSource:0}: Error finding container e332310e72ea67ef5664fd9b52570ce4331381a7431bf92b4bd50d52525694fc: Status 404 returned error can't find the container with id e332310e72ea67ef5664fd9b52570ce4331381a7431bf92b4bd50d52525694fc Jul 10 18:40:12.510028 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.510002 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:12.512343 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:12.512316 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a16c8e1_488a_4082_a279_724ea112e68f.slice/crio-95f1196ee544f619ffadf8f70fedb79414850a7eb189f215193e991af3291572 WatchSource:0}: Error finding container 95f1196ee544f619ffadf8f70fedb79414850a7eb189f215193e991af3291572: Status 404 returned error can't find the container with id 95f1196ee544f619ffadf8f70fedb79414850a7eb189f215193e991af3291572 Jul 10 18:40:12.518752 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.518730 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:12.521062 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:12.521040 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod364b43bc_cd7a_49cb_b214_cdbece4997df.slice/crio-3d3bf63b660bd05cd845b928e2628a299e2ced94c399906adf0443a528b59fe5 WatchSource:0}: Error finding container 3d3bf63b660bd05cd845b928e2628a299e2ced94c399906adf0443a528b59fe5: Status 404 returned error can't find the container with id 3d3bf63b660bd05cd845b928e2628a299e2ced94c399906adf0443a528b59fe5 Jul 10 18:40:12.525424 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.525402 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4xz7g" Jul 10 18:40:12.529097 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:12.529060 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b65b77e_09b9_4937_8014_6431509be7b6.slice/crio-c2c383581c03b7f764607051e212fe8bdc3d59fcd771b2779b8827f5e01b5446 WatchSource:0}: Error finding container c2c383581c03b7f764607051e212fe8bdc3d59fcd771b2779b8827f5e01b5446: Status 404 returned error can't find the container with id c2c383581c03b7f764607051e212fe8bdc3d59fcd771b2779b8827f5e01b5446 Jul 10 18:40:12.540950 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:12.540918 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb1e6736_8a0c_4dcd_bef2_a88218277f25.slice/crio-9baefe6e5aae34f9b68a13c97987e332b8350198ac4ebb5912cdee821fd46c49 WatchSource:0}: Error finding container 9baefe6e5aae34f9b68a13c97987e332b8350198ac4ebb5912cdee821fd46c49: Status 404 returned error can't find the container with id 9baefe6e5aae34f9b68a13c97987e332b8350198ac4ebb5912cdee821fd46c49 Jul 10 18:40:12.545961 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.545937 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:40:12.567605 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.567584 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:40:12.770729 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.770640 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:12.770879 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.770820 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:12.770945 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.770907 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs podName:b22d6861-0651-43b0-8642-f1bb3a2c54ee nodeName:}" failed. No retries permitted until 2026-07-10 18:40:13.770882095 +0000 UTC m=+4.055433408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs") pod "network-metrics-daemon-ccjtn" (UID: "b22d6861-0651-43b0-8642-f1bb3a2c54ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:12.871053 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.871013 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:12.871244 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.871198 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:12.871314 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.871279 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret podName:ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:13.871257523 +0000 UTC m=+4.155808810 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret") pod "global-pull-secret-syncer-dj7wf" (UID: "ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:12.971715 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:12.971681 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr2mx\" (UniqueName: \"kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx\") pod \"network-check-target-prjp9\" (UID: \"b691ab8e-52aa-40ea-9e2b-ce8d242181ef\") " pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:12.971884 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.971852 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:40:12.971924 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.971893 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:40:12.971924 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.971905 2563 projected.go:196] Error preparing data for projected volume kube-api-access-gr2mx for pod openshift-network-diagnostics/network-check-target-prjp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:12.972008 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:12.971969 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx podName:b691ab8e-52aa-40ea-9e2b-ce8d242181ef nodeName:}" failed. No retries permitted until 2026-07-10 18:40:13.971945711 +0000 UTC m=+4.256497000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gr2mx" (UniqueName: "kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx") pod "network-check-target-prjp9" (UID: "b691ab8e-52aa-40ea-9e2b-ce8d242181ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:13.199839 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.199709 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:35:12 +0000 UTC" deadline="2027-12-08 10:27:28.60269678 +0000 UTC" Jul 10 18:40:13.199839 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.199770 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12375h47m15.402930804s" Jul 10 18:40:13.334438 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.334402 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:13.334638 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:13.334540 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:13.335013 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.334987 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:13.335168 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:13.335106 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:13.345546 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.345456 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ltklt" event={"ID":"4b65b77e-09b9-4937-8014-6431509be7b6","Type":"ContainerStarted","Data":"c2c383581c03b7f764607051e212fe8bdc3d59fcd771b2779b8827f5e01b5446"} Jul 10 18:40:13.347797 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.347771 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" event={"ID":"364b43bc-cd7a-49cb-b214-cdbece4997df","Type":"ContainerStarted","Data":"3d3bf63b660bd05cd845b928e2628a299e2ced94c399906adf0443a528b59fe5"} Jul 10 18:40:13.350298 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.350275 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" event={"ID":"3f00c215-ee46-4329-b991-55e2d1a4912f","Type":"ContainerStarted","Data":"80b62478c82d34f06e0ac90c0d7d633ab89039a637a62e0adfc7395c947b62d8"} Jul 10 18:40:13.351757 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.351731 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" event={"ID":"c31d3759-3875-4826-bf22-eafe856a5172","Type":"ContainerStarted","Data":"0110e93b00497000d6c91139cff7abba876d748152112cc60ed03b9f698cd9d1"} Jul 10 18:40:13.353525 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.353490 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xgvw7" event={"ID":"7788073c-5d44-49cd-9ae6-51a11ac16848","Type":"ContainerStarted","Data":"84cd68d2b4d5b98ea3777db7a34ab3ace92023f29097f86653098abf8b7d24da"} Jul 10 18:40:13.355067 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.355043 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4xz7g" event={"ID":"cb1e6736-8a0c-4dcd-bef2-a88218277f25","Type":"ContainerStarted","Data":"9baefe6e5aae34f9b68a13c97987e332b8350198ac4ebb5912cdee821fd46c49"} Jul 10 18:40:13.356692 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.356609 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-brxbd" event={"ID":"4a16c8e1-488a-4082-a279-724ea112e68f","Type":"ContainerStarted","Data":"95f1196ee544f619ffadf8f70fedb79414850a7eb189f215193e991af3291572"} Jul 10 18:40:13.358814 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.358789 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xksn6" event={"ID":"988b88a2-e62e-4676-91e4-01157f069202","Type":"ContainerStarted","Data":"e332310e72ea67ef5664fd9b52570ce4331381a7431bf92b4bd50d52525694fc"} Jul 10 18:40:13.360289 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.360232 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8bbcs" event={"ID":"944afe1e-fa5a-4b53-903e-25329fec3529","Type":"ContainerStarted","Data":"0547e0697f6bef3ff55353392bfc938b825d129922f381fc64865158456b4ce0"} Jul 10 18:40:13.776915 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.776870 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:13.777194 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:13.777147 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:13.777284 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:13.777218 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs podName:b22d6861-0651-43b0-8642-f1bb3a2c54ee nodeName:}" failed. No retries permitted until 2026-07-10 18:40:15.777198185 +0000 UTC m=+6.061749474 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs") pod "network-metrics-daemon-ccjtn" (UID: "b22d6861-0651-43b0-8642-f1bb3a2c54ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:13.877785 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.877747 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:13.877980 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:13.877904 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:13.877980 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:13.877971 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret podName:ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:15.877951143 +0000 UTC m=+6.162502436 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret") pod "global-pull-secret-syncer-dj7wf" (UID: "ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:13.978757 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:13.978706 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr2mx\" (UniqueName: \"kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx\") pod \"network-check-target-prjp9\" (UID: \"b691ab8e-52aa-40ea-9e2b-ce8d242181ef\") " pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:13.978934 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:13.978910 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:40:13.979009 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:13.978939 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:40:13.979009 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:13.978955 2563 projected.go:196] Error preparing data for projected volume kube-api-access-gr2mx for pod openshift-network-diagnostics/network-check-target-prjp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:13.979109 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:13.979025 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx podName:b691ab8e-52aa-40ea-9e2b-ce8d242181ef nodeName:}" failed. No retries permitted until 2026-07-10 18:40:15.979004417 +0000 UTC m=+6.263555722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gr2mx" (UniqueName: "kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx") pod "network-check-target-prjp9" (UID: "b691ab8e-52aa-40ea-9e2b-ce8d242181ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:14.200320 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:14.200224 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:35:12 +0000 UTC" deadline="2028-02-26 04:57:44.325061658 +0000 UTC" Jul 10 18:40:14.200320 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:14.200267 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14290h17m30.124799173s" Jul 10 18:40:14.334914 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:14.334382 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:14.334914 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:14.334519 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:15.334133 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:15.333870 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:15.334608 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:15.334248 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:15.334684 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:15.334644 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:15.334932 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:15.334792 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:15.376278 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:15.376239 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-86.ec2.internal" event={"ID":"f43f038d238bfb0aa554060566ad6e31","Type":"ContainerStarted","Data":"983f6c31a8060781b10910d18c4dab92bd5d3cd8849f40ce7c6d258fcbeca8ad"} Jul 10 18:40:15.794750 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:15.794639 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:15.794970 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:15.794935 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:15.795057 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:15.795015 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs podName:b22d6861-0651-43b0-8642-f1bb3a2c54ee nodeName:}" failed. No retries permitted until 2026-07-10 18:40:19.794985747 +0000 UTC m=+10.079537050 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs") pod "network-metrics-daemon-ccjtn" (UID: "b22d6861-0651-43b0-8642-f1bb3a2c54ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:15.895476 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:15.895438 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:15.895706 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:15.895599 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:15.895706 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:15.895667 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret podName:ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:19.895648008 +0000 UTC m=+10.180199316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret") pod "global-pull-secret-syncer-dj7wf" (UID: "ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:15.996166 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:15.996132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr2mx\" (UniqueName: \"kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx\") pod \"network-check-target-prjp9\" (UID: \"b691ab8e-52aa-40ea-9e2b-ce8d242181ef\") " pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:15.996355 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:15.996327 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:40:15.996418 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:15.996363 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:40:15.996418 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:15.996377 2563 projected.go:196] Error preparing data for projected volume kube-api-access-gr2mx for pod openshift-network-diagnostics/network-check-target-prjp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:15.996523 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:15.996439 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx podName:b691ab8e-52aa-40ea-9e2b-ce8d242181ef nodeName:}" failed. No retries permitted until 2026-07-10 18:40:19.996419101 +0000 UTC m=+10.280970407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gr2mx" (UniqueName: "kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx") pod "network-check-target-prjp9" (UID: "b691ab8e-52aa-40ea-9e2b-ce8d242181ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:16.336381 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:16.335824 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:16.336381 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:16.335980 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:16.381171 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:16.381109 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" event={"ID":"e830fa69557184eff7ed9c13b4c4be24","Type":"ContainerStarted","Data":"24f7de5a5f789e64c4f2bfe196ac1e111eb5aead6ce4579a057b1e215cca0afe"} Jul 10 18:40:16.396678 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:16.396615 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-86.ec2.internal" podStartSLOduration=4.396593426 podStartE2EDuration="4.396593426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:15.39429996 +0000 UTC m=+5.678851268" watchObservedRunningTime="2026-07-10 18:40:16.396593426 +0000 UTC m=+6.681144751" Jul 10 18:40:17.334029 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:17.333992 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:17.334234 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:17.334126 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:17.334765 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:17.334572 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:17.334765 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:17.334681 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:18.334095 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:18.334054 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:18.334604 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:18.334216 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:19.334447 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:19.334411 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:19.334971 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:19.334411 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:19.334971 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:19.334564 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:19.334971 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:19.334609 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:19.829485 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:19.829288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:19.829485 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:19.829420 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:19.829485 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:19.829487 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs podName:b22d6861-0651-43b0-8642-f1bb3a2c54ee nodeName:}" failed. No retries permitted until 2026-07-10 18:40:27.829470142 +0000 UTC m=+18.114021428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs") pod "network-metrics-daemon-ccjtn" (UID: "b22d6861-0651-43b0-8642-f1bb3a2c54ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:19.929734 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:19.929682 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:19.929893 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:19.929840 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:19.929964 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:19.929916 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret podName:ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:27.929895867 +0000 UTC m=+18.214447155 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret") pod "global-pull-secret-syncer-dj7wf" (UID: "ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:20.030379 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:20.030341 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr2mx\" (UniqueName: \"kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx\") pod \"network-check-target-prjp9\" (UID: \"b691ab8e-52aa-40ea-9e2b-ce8d242181ef\") " pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:20.030552 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:20.030530 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:40:20.030611 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:20.030559 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:40:20.030611 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:20.030572 2563 projected.go:196] Error preparing data for projected volume kube-api-access-gr2mx for pod openshift-network-diagnostics/network-check-target-prjp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:20.030721 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:20.030658 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx podName:b691ab8e-52aa-40ea-9e2b-ce8d242181ef nodeName:}" failed. No retries permitted until 2026-07-10 18:40:28.030623165 +0000 UTC m=+18.315174473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gr2mx" (UniqueName: "kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx") pod "network-check-target-prjp9" (UID: "b691ab8e-52aa-40ea-9e2b-ce8d242181ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:20.335254 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:20.335224 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:20.335745 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:20.335332 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:21.334913 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:21.334773 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:21.334913 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:21.334797 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:21.335146 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:21.334915 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:21.335146 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:21.335025 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:21.393672 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:21.393465 2563 generic.go:356] "Generic (PLEG): container finished" podID="e830fa69557184eff7ed9c13b4c4be24" containerID="24f7de5a5f789e64c4f2bfe196ac1e111eb5aead6ce4579a057b1e215cca0afe" exitCode=0 Jul 10 18:40:21.393672 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:21.393518 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" event={"ID":"e830fa69557184eff7ed9c13b4c4be24","Type":"ContainerDied","Data":"24f7de5a5f789e64c4f2bfe196ac1e111eb5aead6ce4579a057b1e215cca0afe"} Jul 10 18:40:22.335259 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:22.334755 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:22.335259 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:22.334911 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:23.334610 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:23.334574 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:23.335053 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:23.334710 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:23.335053 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:23.334580 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:23.335053 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:23.334795 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:24.334382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:24.334346 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:24.334550 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:24.334477 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:25.334805 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:25.334767 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:25.334805 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:25.334803 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:25.335228 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:25.334906 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:25.335228 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:25.335026 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:26.334172 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:26.334129 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:26.334352 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:26.334276 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:27.334747 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:27.334714 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:27.335267 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:27.334714 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:27.335267 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:27.334852 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:27.335267 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:27.334920 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:27.890045 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:27.890006 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:27.890246 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:27.890153 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:27.890246 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:27.890230 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs podName:b22d6861-0651-43b0-8642-f1bb3a2c54ee nodeName:}" failed. No retries permitted until 2026-07-10 18:40:43.890209952 +0000 UTC m=+34.174761248 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs") pod "network-metrics-daemon-ccjtn" (UID: "b22d6861-0651-43b0-8642-f1bb3a2c54ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:27.991003 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:27.990916 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:27.991162 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:27.991050 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:27.991162 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:27.991118 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret podName:ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:43.991098625 +0000 UTC m=+34.275649910 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret") pod "global-pull-secret-syncer-dj7wf" (UID: "ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:28.091615 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:28.091582 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr2mx\" (UniqueName: \"kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx\") pod \"network-check-target-prjp9\" (UID: \"b691ab8e-52aa-40ea-9e2b-ce8d242181ef\") " pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:28.091789 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:28.091753 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:40:28.091789 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:28.091781 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:40:28.091886 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:28.091794 2563 projected.go:196] Error preparing data for projected volume kube-api-access-gr2mx for pod openshift-network-diagnostics/network-check-target-prjp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:28.091886 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:28.091848 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx podName:b691ab8e-52aa-40ea-9e2b-ce8d242181ef nodeName:}" failed. No retries permitted until 2026-07-10 18:40:44.091834282 +0000 UTC m=+34.376385567 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gr2mx" (UniqueName: "kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx") pod "network-check-target-prjp9" (UID: "b691ab8e-52aa-40ea-9e2b-ce8d242181ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:28.334029 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:28.333928 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:28.334184 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:28.334111 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:29.334151 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:29.334120 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:29.334616 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:29.334245 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:29.334616 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:29.334306 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:29.334616 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:29.334423 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:30.335594 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:30.335560 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:30.336008 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:30.335710 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:31.334823 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:31.334775 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:31.335021 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:31.334775 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:31.335021 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:31.334907 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:31.335021 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:31.335002 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:32.334816 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:32.334787 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:32.335236 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:32.334912 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:33.334310 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.333989 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:33.334435 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.334050 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:33.334435 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:33.334391 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:33.334435 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:33.334417 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:33.435562 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.435531 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ltklt" event={"ID":"4b65b77e-09b9-4937-8014-6431509be7b6","Type":"ContainerStarted","Data":"4071bbf7505e31fdae0918e8320c0abfce524232709ac7024162e0de8ddb6e20"} Jul 10 18:40:33.438137 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.438121 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 18:40:33.438523 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.438450 2563 generic.go:356] "Generic (PLEG): container finished" podID="364b43bc-cd7a-49cb-b214-cdbece4997df" containerID="482c75f3fa465b35b6f507b803b7b71547fb1a2df2b817e4ede77c3921db2339" exitCode=1 Jul 10 18:40:33.438523 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.438474 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" event={"ID":"364b43bc-cd7a-49cb-b214-cdbece4997df","Type":"ContainerStarted","Data":"35191a7fb25b2bbca0c74aa989b6f5f753ef20a663d31708a3c74a789e282ef6"} Jul 10 18:40:33.438769 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.438543 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" event={"ID":"364b43bc-cd7a-49cb-b214-cdbece4997df","Type":"ContainerStarted","Data":"481f97bcb9844051860feb7fc8b304ed1717137b93d1b8b6c1a05ddf150b6ec6"} Jul 10 18:40:33.438769 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.438579 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" event={"ID":"364b43bc-cd7a-49cb-b214-cdbece4997df","Type":"ContainerStarted","Data":"115b68694b67833cd96aae13991cd50c65a2378257194df154ace1af025cd521"} Jul 10 18:40:33.438769 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.438623 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" event={"ID":"364b43bc-cd7a-49cb-b214-cdbece4997df","Type":"ContainerStarted","Data":"496b6f88640b095891470d6cc9d6aa04590dc1b8cb9ba524808ecd3bf480acc3"} Jul 10 18:40:33.438769 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.438688 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" event={"ID":"364b43bc-cd7a-49cb-b214-cdbece4997df","Type":"ContainerDied","Data":"482c75f3fa465b35b6f507b803b7b71547fb1a2df2b817e4ede77c3921db2339"} Jul 10 18:40:33.438769 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.438738 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" event={"ID":"364b43bc-cd7a-49cb-b214-cdbece4997df","Type":"ContainerStarted","Data":"401826f9939afc146bd37267df89b0e1cfae578bad80ba005eb4dfa511bdfdaf"} Jul 10 18:40:33.439915 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.439899 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" event={"ID":"3f00c215-ee46-4329-b991-55e2d1a4912f","Type":"ContainerStarted","Data":"4abec1419febb096912f789a5d88938d82b7fa231ca53762ac9b30e6667a29c5"} Jul 10 18:40:33.441125 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.441106 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" event={"ID":"c31d3759-3875-4826-bf22-eafe856a5172","Type":"ContainerStarted","Data":"7be200830f722283b9e33e90eb8b248792c7e90481ff522256a5788d9f9c0551"} Jul 10 18:40:33.442538 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.442519 2563 generic.go:356] "Generic (PLEG): container finished" podID="7788073c-5d44-49cd-9ae6-51a11ac16848" containerID="f9becf3af27c5cf587714343541a22ddef30861328e668a7810f127dc22f09b1" exitCode=0 Jul 10 18:40:33.442631 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.442580 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xgvw7" event={"ID":"7788073c-5d44-49cd-9ae6-51a11ac16848","Type":"ContainerDied","Data":"f9becf3af27c5cf587714343541a22ddef30861328e668a7810f127dc22f09b1"} Jul 10 18:40:33.444278 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.444259 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" event={"ID":"e830fa69557184eff7ed9c13b4c4be24","Type":"ContainerStarted","Data":"32f96e1fbf3afac5b34b6a132ef1fd688e24d1941ba6bb834a7012eb4c43873e"} Jul 10 18:40:33.445766 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.445745 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4xz7g" event={"ID":"cb1e6736-8a0c-4dcd-bef2-a88218277f25","Type":"ContainerStarted","Data":"04a87c1f81d36f009f1cc9448ca210afe0b50ca119517929fde732e86a8ada9a"} Jul 10 18:40:33.447130 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.447105 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-brxbd" event={"ID":"4a16c8e1-488a-4082-a279-724ea112e68f","Type":"ContainerStarted","Data":"bf10b95ee3938599ed162eb57f936a24ef66c443bf2c2cbe68402e17c4953799"} Jul 10 18:40:33.448485 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.448461 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xksn6" event={"ID":"988b88a2-e62e-4676-91e4-01157f069202","Type":"ContainerStarted","Data":"694159a126437f3ccfa31817e1c736565e681303d6704fc21694fbc005706d8a"} Jul 10 18:40:33.463951 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.463916 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ltklt" podStartSLOduration=2.73046989 podStartE2EDuration="22.463904768s" totalImagesPullingTime="19.733434878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:33.450241578 +0000 UTC m=+23.734792884" watchObservedRunningTime="2026-07-10 18:40:33.463904768 +0000 UTC m=+23.748456078" Jul 10 18:40:33.464048 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.464008 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xksn6" podStartSLOduration=2.7176673019999997 podStartE2EDuration="22.464004298s" totalImagesPullingTime="19.746336996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:33.463661165 +0000 UTC m=+23.748212497" watchObservedRunningTime="2026-07-10 18:40:33.464004298 +0000 UTC m=+23.748555604" Jul 10 18:40:33.481965 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.481924 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-brxbd" podStartSLOduration=2.717728427 podStartE2EDuration="22.481913413s" totalImagesPullingTime="19.764184986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:33.481686567 +0000 UTC m=+23.766237884" watchObservedRunningTime="2026-07-10 18:40:33.481913413 +0000 UTC m=+23.766464719" Jul 10 18:40:33.497715 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.497682 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4xz7g" podStartSLOduration=2.773515574 podStartE2EDuration="22.497672412s" totalImagesPullingTime="19.724156838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:33.497208926 +0000 UTC m=+23.781760233" watchObservedRunningTime="2026-07-10 18:40:33.497672412 +0000 UTC m=+23.782223715" Jul 10 18:40:33.569452 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.569415 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jwrjq" podStartSLOduration=2.775897865 podStartE2EDuration="22.569404188s" totalImagesPullingTime="19.793506323s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:33.568952736 +0000 UTC m=+23.853504042" watchObservedRunningTime="2026-07-10 18:40:33.569404188 +0000 UTC m=+23.853955494" Jul 10 18:40:33.569560 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.569521 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-86.ec2.internal" podStartSLOduration=21.569513627 podStartE2EDuration="21.569513627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:33.546573634 +0000 UTC m=+23.831124939" watchObservedRunningTime="2026-07-10 18:40:33.569513627 +0000 UTC m=+23.854064937" Jul 10 18:40:33.577622 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.577600 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:33.725426 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:33.725393 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 18:40:34.220363 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:34.220178 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T18:40:33.725406116Z","UUID":"ce1d5a09-8e98-4412-b103-7c146f585692","Handler":null,"Name":"","Endpoint":""} Jul 10 18:40:34.222207 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:34.222183 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 18:40:34.222336 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:34.222230 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 18:40:34.334350 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:34.334322 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:34.334525 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:34.334435 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:34.453682 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:34.453635 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" event={"ID":"3f00c215-ee46-4329-b991-55e2d1a4912f","Type":"ContainerStarted","Data":"787e79788cda18360a25e9cbb444734d26537df15defa6bd9a357adb03aa6a8f"} Jul 10 18:40:34.455279 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:34.455246 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8bbcs" event={"ID":"944afe1e-fa5a-4b53-903e-25329fec3529","Type":"ContainerStarted","Data":"1e0ebedce691966251cf24489745086ac75122c3b635334000db8406acde6f01"} Jul 10 18:40:34.476237 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:34.476147 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8bbcs" podStartSLOduration=3.698165078 podStartE2EDuration="23.476127787s" totalImagesPullingTime="19.777962709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:34.475800565 +0000 UTC m=+24.760351886" watchObservedRunningTime="2026-07-10 18:40:34.476127787 +0000 UTC m=+24.760679097" Jul 10 18:40:35.062562 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:35.062489 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:35.063244 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:35.063221 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:35.334403 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:35.334377 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:35.334554 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:35.334409 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:35.334554 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:35.334512 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:35.334683 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:35.334640 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:35.461166 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:35.461140 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 18:40:35.461661 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:35.461619 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" event={"ID":"364b43bc-cd7a-49cb-b214-cdbece4997df","Type":"ContainerStarted","Data":"45f58b9f4327ee965ace60c2ab33656b6a3fa84f4076ea2d788c15eed8e5f574"} Jul 10 18:40:35.463883 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:35.463854 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" event={"ID":"3f00c215-ee46-4329-b991-55e2d1a4912f","Type":"ContainerStarted","Data":"e05a65344dd04e2edcb3821ad3836a40d61a50aa8470a676c4988c036efb7746"} Jul 10 18:40:35.839522 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:35.839346 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:36.334062 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:36.333999 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:36.334248 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:36.334112 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:36.466170 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:36.466135 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 18:40:37.334001 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:37.333975 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:37.334181 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:37.334011 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:37.334181 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:37.334126 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:37.334266 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:37.334183 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:38.334118 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.333951 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:38.334846 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:38.334219 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:38.472930 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.472908 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 18:40:38.473257 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.473236 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" event={"ID":"364b43bc-cd7a-49cb-b214-cdbece4997df","Type":"ContainerStarted","Data":"01d4620901e7611838c943a4b63e3b9365f94e94b819a3c7639f750df7205c73"} Jul 10 18:40:38.476480 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.474331 2563 scope.go:117] "RemoveContainer" containerID="482c75f3fa465b35b6f507b803b7b71547fb1a2df2b817e4ede77c3921db2339" Jul 10 18:40:38.476480 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.474593 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:38.476480 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.474720 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:38.476480 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.474785 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:38.478400 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.478376 2563 generic.go:356] "Generic (PLEG): container finished" podID="7788073c-5d44-49cd-9ae6-51a11ac16848" containerID="5a46359c31259e0a63edf65c1325b775970e27e32e6c8a319a708513a832c1ca" exitCode=0 Jul 10 18:40:38.478504 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.478412 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xgvw7" event={"ID":"7788073c-5d44-49cd-9ae6-51a11ac16848","Type":"ContainerDied","Data":"5a46359c31259e0a63edf65c1325b775970e27e32e6c8a319a708513a832c1ca"} Jul 10 18:40:38.492915 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.492893 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:38.493228 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.493205 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:40:38.505446 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:38.505407 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p66bw" podStartSLOduration=5.683984909 podStartE2EDuration="27.50539437s" totalImagesPullingTime="21.821409461s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:35.497389816 +0000 UTC m=+25.781941124" watchObservedRunningTime="2026-07-10 18:40:38.50539437 +0000 UTC m=+28.789945667" Jul 10 18:40:39.334606 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:39.334570 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:39.335284 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:39.334570 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:39.335284 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:39.334719 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:39.335284 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:39.334798 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:39.484845 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:39.484803 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 18:40:39.485251 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:39.485221 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" event={"ID":"364b43bc-cd7a-49cb-b214-cdbece4997df","Type":"ContainerStarted","Data":"8e74272f9ae6830499283458f46978e839337c5d463fecaeb651b42ff56c0acc"} Jul 10 18:40:39.515178 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:39.515138 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" podStartSLOduration=8.510271107 podStartE2EDuration="28.515124061s" totalImagesPullingTime="20.004852954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:39.514708163 +0000 UTC m=+29.799259498" watchObservedRunningTime="2026-07-10 18:40:39.515124061 +0000 UTC m=+29.799675368" Jul 10 18:40:39.587845 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:39.587778 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-prjp9"] Jul 10 18:40:39.587962 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:39.587865 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:39.588002 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:39.587985 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:39.589080 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:39.589058 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dj7wf"] Jul 10 18:40:39.589160 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:39.589148 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:39.589237 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:39.589218 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:39.592434 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:39.592409 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ccjtn"] Jul 10 18:40:39.592630 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:39.592495 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:39.592630 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:39.592585 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:40.489898 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:40.489732 2563 generic.go:356] "Generic (PLEG): container finished" podID="7788073c-5d44-49cd-9ae6-51a11ac16848" containerID="9542cffe041372768a621ff0934fdd95f333ac63fd20835f127797a02c57c061" exitCode=0 Jul 10 18:40:40.490309 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:40.489813 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xgvw7" event={"ID":"7788073c-5d44-49cd-9ae6-51a11ac16848","Type":"ContainerDied","Data":"9542cffe041372768a621ff0934fdd95f333ac63fd20835f127797a02c57c061"} Jul 10 18:40:41.333799 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:41.333772 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:41.333986 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:41.333777 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:41.333986 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:41.333863 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:41.333986 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:41.333781 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:41.333986 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:41.333946 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:41.334139 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:41.333996 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:42.202320 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:42.202279 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:42.202976 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:42.202466 2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 18:40:42.203485 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:42.203376 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ltklt" Jul 10 18:40:42.267266 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:42.267238 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:42.496831 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:42.496767 2563 generic.go:356] "Generic (PLEG): container finished" podID="7788073c-5d44-49cd-9ae6-51a11ac16848" containerID="6673d0f30e6424827c76b489b3f2cf5b3609ded3906d94b9b9774de2fba6eb1b" exitCode=0 Jul 10 18:40:42.496947 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:42.496853 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xgvw7" event={"ID":"7788073c-5d44-49cd-9ae6-51a11ac16848","Type":"ContainerDied","Data":"6673d0f30e6424827c76b489b3f2cf5b3609ded3906d94b9b9774de2fba6eb1b"} Jul 10 18:40:43.334375 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:43.334345 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:43.334829 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:43.334345 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:43.334829 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:43.334474 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjtn" podUID="b22d6861-0651-43b0-8642-f1bb3a2c54ee" Jul 10 18:40:43.334829 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:43.334526 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-prjp9" podUID="b691ab8e-52aa-40ea-9e2b-ce8d242181ef" Jul 10 18:40:43.334829 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:43.334345 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:43.334829 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:43.334608 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dj7wf" podUID="ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1" Jul 10 18:40:43.914685 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:43.914633 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:43.914948 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:43.914754 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:43.914948 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:43.914828 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs podName:b22d6861-0651-43b0-8642-f1bb3a2c54ee nodeName:}" failed. No retries permitted until 2026-07-10 18:41:15.914805696 +0000 UTC m=+66.199356982 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs") pod "network-metrics-daemon-ccjtn" (UID: "b22d6861-0651-43b0-8642-f1bb3a2c54ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:40:44.015626 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:44.015600 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:44.015792 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:44.015728 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:44.015792 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:44.015784 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret podName:ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:16.015766935 +0000 UTC m=+66.300318235 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret") pod "global-pull-secret-syncer-dj7wf" (UID: "ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:40:44.116024 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:44.115993 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr2mx\" (UniqueName: \"kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx\") pod \"network-check-target-prjp9\" (UID: \"b691ab8e-52aa-40ea-9e2b-ce8d242181ef\") " pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:44.116187 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:44.116136 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:40:44.116187 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:44.116152 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:40:44.116187 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:44.116162 2563 projected.go:196] Error preparing data for projected volume kube-api-access-gr2mx for pod openshift-network-diagnostics/network-check-target-prjp9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:44.116320 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:44.116222 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx podName:b691ab8e-52aa-40ea-9e2b-ce8d242181ef nodeName:}" failed. No retries permitted until 2026-07-10 18:41:16.116205412 +0000 UTC m=+66.400756733 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gr2mx" (UniqueName: "kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx") pod "network-check-target-prjp9" (UID: "b691ab8e-52aa-40ea-9e2b-ce8d242181ef") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:40:45.070841 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.070771 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-86.ec2.internal" event="NodeReady" Jul 10 18:40:45.071345 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.070892 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 18:40:45.134288 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.134261 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-brwtp"] Jul 10 18:40:45.136329 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.136311 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w9rnj"] Jul 10 18:40:45.136481 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.136452 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.138087 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.138069 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w9rnj" Jul 10 18:40:45.139236 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.139196 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 18:40:45.139568 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.139548 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 18:40:45.139673 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.139568 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s4tdx\"" Jul 10 18:40:45.140618 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.140475 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 18:40:45.140618 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.140497 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 18:40:45.140776 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.140623 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q5td4\"" Jul 10 18:40:45.140828 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.140818 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 18:40:45.151795 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.151740 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w9rnj"] Jul 10 18:40:45.152791 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.152525 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-brwtp"] Jul 10 18:40:45.223544 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.223523 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e89500b-d296-4aa1-afd5-359d27ba0a38-tmp-dir\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.223681 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.223574 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwzq2\" (UniqueName: \"kubernetes.io/projected/9b74d98f-b159-4df8-9a21-79981099cc0e-kube-api-access-rwzq2\") pod \"ingress-canary-w9rnj\" (UID: \"9b74d98f-b159-4df8-9a21-79981099cc0e\") " pod="openshift-ingress-canary/ingress-canary-w9rnj" Jul 10 18:40:45.223681 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.223598 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.223681 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.223630 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b74d98f-b159-4df8-9a21-79981099cc0e-cert\") pod \"ingress-canary-w9rnj\" (UID: \"9b74d98f-b159-4df8-9a21-79981099cc0e\") " pod="openshift-ingress-canary/ingress-canary-w9rnj" Jul 10 18:40:45.223828 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.223679 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e89500b-d296-4aa1-afd5-359d27ba0a38-config-volume\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.223828 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.223703 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9p77\" (UniqueName: \"kubernetes.io/projected/1e89500b-d296-4aa1-afd5-359d27ba0a38-kube-api-access-s9p77\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.319161 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.319135 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:45.324676 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.324609 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b74d98f-b159-4df8-9a21-79981099cc0e-cert\") pod \"ingress-canary-w9rnj\" (UID: \"9b74d98f-b159-4df8-9a21-79981099cc0e\") " pod="openshift-ingress-canary/ingress-canary-w9rnj" Jul 10 18:40:45.324676 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.324667 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e89500b-d296-4aa1-afd5-359d27ba0a38-config-volume\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.324816 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.324699 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9p77\" (UniqueName: \"kubernetes.io/projected/1e89500b-d296-4aa1-afd5-359d27ba0a38-kube-api-access-s9p77\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.324816 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.324736 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e89500b-d296-4aa1-afd5-359d27ba0a38-tmp-dir\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.324950 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.324929 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwzq2\" (UniqueName: \"kubernetes.io/projected/9b74d98f-b159-4df8-9a21-79981099cc0e-kube-api-access-rwzq2\") pod \"ingress-canary-w9rnj\" (UID: \"9b74d98f-b159-4df8-9a21-79981099cc0e\") " pod="openshift-ingress-canary/ingress-canary-w9rnj" Jul 10 18:40:45.325023 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.324965 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.325090 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.325060 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e89500b-d296-4aa1-afd5-359d27ba0a38-tmp-dir\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.325090 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:45.325066 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:45.325189 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:45.325127 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls podName:1e89500b-d296-4aa1-afd5-359d27ba0a38 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:45.825109679 +0000 UTC m=+36.109660969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls") pod "dns-default-brwtp" (UID: "1e89500b-d296-4aa1-afd5-359d27ba0a38") : secret "dns-default-metrics-tls" not found Jul 10 18:40:45.325579 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.325389 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e89500b-d296-4aa1-afd5-359d27ba0a38-config-volume\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.329323 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.329307 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b74d98f-b159-4df8-9a21-79981099cc0e-cert\") pod \"ingress-canary-w9rnj\" (UID: \"9b74d98f-b159-4df8-9a21-79981099cc0e\") " pod="openshift-ingress-canary/ingress-canary-w9rnj" Jul 10 18:40:45.334139 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.334117 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:40:45.334282 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.334244 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:40:45.334282 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.334272 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:40:45.337839 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.337811 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:40:45.337964 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.337810 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 18:40:45.338054 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.338023 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-crdnw\"" Jul 10 18:40:45.338297 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.338273 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:40:45.338368 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.338273 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-75lnr\"" Jul 10 18:40:45.338368 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.338311 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 18:40:45.338959 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.338937 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9p77\" (UniqueName: \"kubernetes.io/projected/1e89500b-d296-4aa1-afd5-359d27ba0a38-kube-api-access-s9p77\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.343939 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.343918 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwzq2\" (UniqueName: \"kubernetes.io/projected/9b74d98f-b159-4df8-9a21-79981099cc0e-kube-api-access-rwzq2\") pod \"ingress-canary-w9rnj\" (UID: \"9b74d98f-b159-4df8-9a21-79981099cc0e\") " pod="openshift-ingress-canary/ingress-canary-w9rnj" Jul 10 18:40:45.425800 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.425771 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:45.455721 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.455700 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w9rnj" Jul 10 18:40:45.594107 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.594015 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w9rnj"] Jul 10 18:40:45.828874 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:45.828837 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:45.829042 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:45.829009 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:45.829123 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:45.829110 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls podName:1e89500b-d296-4aa1-afd5-359d27ba0a38 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:46.829087009 +0000 UTC m=+37.113638298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls") pod "dns-default-brwtp" (UID: "1e89500b-d296-4aa1-afd5-359d27ba0a38") : secret "dns-default-metrics-tls" not found Jul 10 18:40:46.564953 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:46.564916 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:46.631928 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:46.631884 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:46.838218 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:46.838131 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:46.838376 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:46.838309 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:46.838434 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:46.838386 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls podName:1e89500b-d296-4aa1-afd5-359d27ba0a38 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:48.838367514 +0000 UTC m=+39.122918821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls") pod "dns-default-brwtp" (UID: "1e89500b-d296-4aa1-afd5-359d27ba0a38") : secret "dns-default-metrics-tls" not found Jul 10 18:40:47.511263 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:47.511167 2563 generic.go:356] "Generic (PLEG): container finished" podID="7788073c-5d44-49cd-9ae6-51a11ac16848" containerID="d3bbcdcd364074da5b47d0301e32b157d98318544f1ddd24faadae78977e8c0d" exitCode=0 Jul 10 18:40:47.511425 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:47.511266 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xgvw7" event={"ID":"7788073c-5d44-49cd-9ae6-51a11ac16848","Type":"ContainerDied","Data":"d3bbcdcd364074da5b47d0301e32b157d98318544f1ddd24faadae78977e8c0d"} Jul 10 18:40:47.512349 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:47.512325 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w9rnj" event={"ID":"9b74d98f-b159-4df8-9a21-79981099cc0e","Type":"ContainerStarted","Data":"b0dd7a0a9e016582104947099c67ef7440facf020c353159f30ab6eec9152705"} Jul 10 18:40:48.519081 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:48.518828 2563 generic.go:356] "Generic (PLEG): container finished" podID="7788073c-5d44-49cd-9ae6-51a11ac16848" containerID="b1f4818998c8b84ce347bb3a1ebe79da319bbb9e6d9e8f8149ecbd3f83243305" exitCode=0 Jul 10 18:40:48.519081 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:48.518922 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xgvw7" event={"ID":"7788073c-5d44-49cd-9ae6-51a11ac16848","Type":"ContainerDied","Data":"b1f4818998c8b84ce347bb3a1ebe79da319bbb9e6d9e8f8149ecbd3f83243305"} Jul 10 18:40:48.852913 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:48.852828 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:48.853048 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:48.852990 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:48.853104 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:48.853051 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls podName:1e89500b-d296-4aa1-afd5-359d27ba0a38 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:52.853034392 +0000 UTC m=+43.137585677 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls") pod "dns-default-brwtp" (UID: "1e89500b-d296-4aa1-afd5-359d27ba0a38") : secret "dns-default-metrics-tls" not found Jul 10 18:40:49.525945 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:49.525864 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xgvw7" event={"ID":"7788073c-5d44-49cd-9ae6-51a11ac16848","Type":"ContainerStarted","Data":"a8ad48b9bd2c4e2310435ccdd062a7a40584ee05c0abab041feba564d9d44d05"} Jul 10 18:40:49.548785 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:49.548736 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xgvw7" podStartSLOduration=8.31453665 podStartE2EDuration="39.54872191s" totalImagesPullingTime="31.23418526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:49.548573185 +0000 UTC m=+39.833124504" watchObservedRunningTime="2026-07-10 18:40:49.54872191 +0000 UTC m=+39.833273218" Jul 10 18:40:49.679738 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:49.679700 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:49.747304 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:49.747256 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:50.058686 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:50.058637 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:50.122310 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:50.122275 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:50.529351 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:50.529314 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w9rnj" event={"ID":"9b74d98f-b159-4df8-9a21-79981099cc0e","Type":"ContainerStarted","Data":"850b1632c16fce9e47c9e93584f45671c3df6392e770ad1cc4e401871900749c"} Jul 10 18:40:50.546605 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:50.546561 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w9rnj" podStartSLOduration=2.327434988 podStartE2EDuration="5.546546558s" totalImagesPullingTime="3.21911157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:50.546009782 +0000 UTC m=+40.830561091" watchObservedRunningTime="2026-07-10 18:40:50.546546558 +0000 UTC m=+40.831097899" Jul 10 18:40:52.881938 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:52.881903 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:40:52.882289 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:52.882027 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:40:52.882289 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:52.882077 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls podName:1e89500b-d296-4aa1-afd5-359d27ba0a38 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:00.882064113 +0000 UTC m=+51.166615399 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls") pod "dns-default-brwtp" (UID: "1e89500b-d296-4aa1-afd5-359d27ba0a38") : secret "dns-default-metrics-tls" not found Jul 10 18:40:54.650344 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.650312 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k5htc"] Jul 10 18:40:54.682462 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.682434 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k5htc"] Jul 10 18:40:54.682584 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.682520 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k5htc" Jul 10 18:40:54.684962 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.684936 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 10 18:40:54.685061 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.684939 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:40:54.685117 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.685107 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-wfrkf\"" Jul 10 18:40:54.794064 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.794032 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwwbr\" (UniqueName: \"kubernetes.io/projected/da9b2366-b93a-4e09-b45b-c974c829e26d-kube-api-access-dwwbr\") pod \"volume-data-source-validator-78495597f6-k5htc\" (UID: \"da9b2366-b93a-4e09-b45b-c974c829e26d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k5htc" Jul 10 18:40:54.871055 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.871026 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8"] Jul 10 18:40:54.886832 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.886809 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8"] Jul 10 18:40:54.886950 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.886901 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:40:54.890120 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.890100 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 10 18:40:54.890253 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.890238 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:40:54.890352 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.890339 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 10 18:40:54.890660 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.890628 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-52hbf\"" Jul 10 18:40:54.895294 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.895274 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwwbr\" (UniqueName: \"kubernetes.io/projected/da9b2366-b93a-4e09-b45b-c974c829e26d-kube-api-access-dwwbr\") pod \"volume-data-source-validator-78495597f6-k5htc\" (UID: \"da9b2366-b93a-4e09-b45b-c974c829e26d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k5htc" Jul 10 18:40:54.909257 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.909208 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwwbr\" (UniqueName: \"kubernetes.io/projected/da9b2366-b93a-4e09-b45b-c974c829e26d-kube-api-access-dwwbr\") pod \"volume-data-source-validator-78495597f6-k5htc\" (UID: \"da9b2366-b93a-4e09-b45b-c974c829e26d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k5htc" Jul 10 18:40:54.923342 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.923318 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:54.985047 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.985020 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:54.990842 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.990823 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k5htc" Jul 10 18:40:54.996138 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.996118 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-vvdl8\" (UID: \"0be8606b-b59d-4412-b7fe-7f994e872ef4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:40:54.996214 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:54.996164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vm7g\" (UniqueName: \"kubernetes.io/projected/0be8606b-b59d-4412-b7fe-7f994e872ef4-kube-api-access-5vm7g\") pod \"cluster-samples-operator-65db858894-vvdl8\" (UID: \"0be8606b-b59d-4412-b7fe-7f994e872ef4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:40:55.096950 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:55.096926 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-vvdl8\" (UID: \"0be8606b-b59d-4412-b7fe-7f994e872ef4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:40:55.097085 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:55.096977 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vm7g\" (UniqueName: \"kubernetes.io/projected/0be8606b-b59d-4412-b7fe-7f994e872ef4-kube-api-access-5vm7g\") pod \"cluster-samples-operator-65db858894-vvdl8\" (UID: \"0be8606b-b59d-4412-b7fe-7f994e872ef4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:40:55.097125 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:55.097088 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:40:55.097165 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:55.097155 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls podName:0be8606b-b59d-4412-b7fe-7f994e872ef4 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:55.597136924 +0000 UTC m=+45.881688209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls") pod "cluster-samples-operator-65db858894-vvdl8" (UID: "0be8606b-b59d-4412-b7fe-7f994e872ef4") : secret "samples-operator-tls" not found Jul 10 18:40:55.106869 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:55.106839 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vm7g\" (UniqueName: \"kubernetes.io/projected/0be8606b-b59d-4412-b7fe-7f994e872ef4-kube-api-access-5vm7g\") pod \"cluster-samples-operator-65db858894-vvdl8\" (UID: \"0be8606b-b59d-4412-b7fe-7f994e872ef4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:40:55.121091 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:55.121047 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k5htc"] Jul 10 18:40:55.124194 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:40:55.124173 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda9b2366_b93a_4e09_b45b_c974c829e26d.slice/crio-260337fb6bbbc5cbcfedfc8630363ed9458537a420a91322c1d1cc660dd22dce WatchSource:0}: Error finding container 260337fb6bbbc5cbcfedfc8630363ed9458537a420a91322c1d1cc660dd22dce: Status 404 returned error can't find the container with id 260337fb6bbbc5cbcfedfc8630363ed9458537a420a91322c1d1cc660dd22dce Jul 10 18:40:55.541951 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:55.541920 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k5htc" event={"ID":"da9b2366-b93a-4e09-b45b-c974c829e26d","Type":"ContainerStarted","Data":"260337fb6bbbc5cbcfedfc8630363ed9458537a420a91322c1d1cc660dd22dce"} Jul 10 18:40:55.600860 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:55.600829 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-vvdl8\" (UID: \"0be8606b-b59d-4412-b7fe-7f994e872ef4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:40:55.600996 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:55.600977 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:40:55.601045 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:55.601039 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls podName:0be8606b-b59d-4412-b7fe-7f994e872ef4 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:56.601021415 +0000 UTC m=+46.885572700 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls") pod "cluster-samples-operator-65db858894-vvdl8" (UID: "0be8606b-b59d-4412-b7fe-7f994e872ef4") : secret "samples-operator-tls" not found Jul 10 18:40:56.607477 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:56.607451 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-vvdl8\" (UID: \"0be8606b-b59d-4412-b7fe-7f994e872ef4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:40:56.607824 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:56.607581 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:40:56.607824 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:56.607675 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls podName:0be8606b-b59d-4412-b7fe-7f994e872ef4 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:58.607629556 +0000 UTC m=+48.892180842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls") pod "cluster-samples-operator-65db858894-vvdl8" (UID: "0be8606b-b59d-4412-b7fe-7f994e872ef4") : secret "samples-operator-tls" not found Jul 10 18:40:57.547660 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:57.547611 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k5htc" event={"ID":"da9b2366-b93a-4e09-b45b-c974c829e26d","Type":"ContainerStarted","Data":"81c031fc383776e2747defc0f07f704275b29b6a70dbcb21df1ada2dcb7f0508"} Jul 10 18:40:57.564192 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:57.564147 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-k5htc" podStartSLOduration=2.1379165589999998 podStartE2EDuration="3.564133293s" totalImagesPullingTime="1.426216734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:40:57.562889028 +0000 UTC m=+47.847440335" watchObservedRunningTime="2026-07-10 18:40:57.564133293 +0000 UTC m=+47.848684638" Jul 10 18:40:57.678749 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:57.678709 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:57.947354 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:57.947322 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69d5f554d8-jf4wt"] Jul 10 18:40:57.978495 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:57.978470 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69d5f554d8-jf4wt"] Jul 10 18:40:57.978617 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:57.978558 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:57.981046 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:57.981026 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 18:40:57.981046 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:57.981036 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 18:40:57.981046 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:57.981046 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 18:40:57.981235 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:57.981035 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tn8fv\"" Jul 10 18:40:57.989527 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:57.989510 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 18:40:58.118263 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.118234 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-installation-pull-secrets\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.118410 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.118269 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-trusted-ca\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.118410 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.118306 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-image-registry-private-configuration\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.118410 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.118360 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-certificates\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.118410 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.118395 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.118565 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.118430 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pjhp\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-kube-api-access-4pjhp\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.118565 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.118462 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baf2fb14-f51e-43cc-a8af-a5fc371113c8-ca-trust-extracted\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.118565 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.118510 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-bound-sa-token\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.219097 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.219033 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-installation-pull-secrets\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.219097 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.219065 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-trusted-ca\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.219285 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.219099 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-image-registry-private-configuration\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.219285 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.219126 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-certificates\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.219285 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.219156 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.219285 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.219210 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pjhp\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-kube-api-access-4pjhp\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.219285 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.219244 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baf2fb14-f51e-43cc-a8af-a5fc371113c8-ca-trust-extracted\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.219285 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.219273 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-bound-sa-token\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.219550 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:58.219321 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:40:58.219550 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:58.219343 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d5f554d8-jf4wt: secret "image-registry-tls" not found Jul 10 18:40:58.219550 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:58.219412 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls podName:baf2fb14-f51e-43cc-a8af-a5fc371113c8 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:58.719388231 +0000 UTC m=+49.003939543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls") pod "image-registry-69d5f554d8-jf4wt" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8") : secret "image-registry-tls" not found Jul 10 18:40:58.219834 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.219817 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baf2fb14-f51e-43cc-a8af-a5fc371113c8-ca-trust-extracted\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.220072 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.220049 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-certificates\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.220192 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.220177 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-trusted-ca\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.221361 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.221333 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-installation-pull-secrets\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.221445 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.221382 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-image-registry-private-configuration\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.232535 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.232512 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pjhp\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-kube-api-access-4pjhp\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.234514 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.234491 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-bound-sa-token\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.623606 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.623520 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-vvdl8\" (UID: \"0be8606b-b59d-4412-b7fe-7f994e872ef4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:40:58.623777 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:58.623695 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:40:58.623777 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:58.623758 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls podName:0be8606b-b59d-4412-b7fe-7f994e872ef4 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:02.623741314 +0000 UTC m=+52.908292599 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls") pod "cluster-samples-operator-65db858894-vvdl8" (UID: "0be8606b-b59d-4412-b7fe-7f994e872ef4") : secret "samples-operator-tls" not found Jul 10 18:40:58.724600 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:58.724571 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:58.725069 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:58.724746 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:40:58.725069 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:58.724762 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d5f554d8-jf4wt: secret "image-registry-tls" not found Jul 10 18:40:58.725069 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:58.724810 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls podName:baf2fb14-f51e-43cc-a8af-a5fc371113c8 nodeName:}" failed. No retries permitted until 2026-07-10 18:40:59.724796864 +0000 UTC m=+50.009348150 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls") pod "image-registry-69d5f554d8-jf4wt" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8") : secret "image-registry-tls" not found Jul 10 18:40:59.430078 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.430044 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8wf7f"] Jul 10 18:40:59.461316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.461287 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8wf7f"] Jul 10 18:40:59.461444 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.461372 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.470532 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.470509 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 18:40:59.470629 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.470512 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 18:40:59.470689 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.470670 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 18:40:59.471469 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.471453 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 18:40:59.471517 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.471453 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m5ckx\"" Jul 10 18:40:59.587905 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.587851 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:59.631920 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.631889 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxkkh\" (UniqueName: \"kubernetes.io/projected/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-api-access-bxkkh\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.632078 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.631930 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.632078 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.631962 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a220158e-f493-448c-8c20-0b3c640fc0d9-data-volume\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.632078 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.632015 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a220158e-f493-448c-8c20-0b3c640fc0d9-crio-socket\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.632078 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.632057 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.698950 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.698860 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:59.733208 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.733176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxkkh\" (UniqueName: \"kubernetes.io/projected/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-api-access-bxkkh\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.733221 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.733256 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a220158e-f493-448c-8c20-0b3c640fc0d9-data-volume\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.733275 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a220158e-f493-448c-8c20-0b3c640fc0d9-crio-socket\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.733293 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.733342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:59.733421 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:59.733451 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:59.733465 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d5f554d8-jf4wt: secret "image-registry-tls" not found Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:59.733498 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm podName:a220158e-f493-448c-8c20-0b3c640fc0d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:00.23347936 +0000 UTC m=+50.518030667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8wf7f" (UID: "a220158e-f493-448c-8c20-0b3c640fc0d9") : configmap "kube-rbac-proxy" not found Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.733494 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a220158e-f493-448c-8c20-0b3c640fc0d9-crio-socket\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:59.733424 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:59.733518 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls podName:baf2fb14-f51e-43cc-a8af-a5fc371113c8 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:01.733508421 +0000 UTC m=+52.018059714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls") pod "image-registry-69d5f554d8-jf4wt" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8") : secret "image-registry-tls" not found Jul 10 18:40:59.733591 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:40:59.733561 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls podName:a220158e-f493-448c-8c20-0b3c640fc0d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:00.233552588 +0000 UTC m=+50.518103872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8wf7f" (UID: "a220158e-f493-448c-8c20-0b3c640fc0d9") : secret "insights-runtime-extractor-tls" not found Jul 10 18:40:59.734022 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.733603 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a220158e-f493-448c-8c20-0b3c640fc0d9-data-volume\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:40:59.741891 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:40:59.741869 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxkkh\" (UniqueName: \"kubernetes.io/projected/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-api-access-bxkkh\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:00.237713 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:00.237684 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:00.237873 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:00.237733 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:00.237873 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:00.237822 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:41:00.237873 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:00.237830 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:41:00.238019 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:00.237902 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls podName:a220158e-f493-448c-8c20-0b3c640fc0d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:01.237883994 +0000 UTC m=+51.522435299 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8wf7f" (UID: "a220158e-f493-448c-8c20-0b3c640fc0d9") : secret "insights-runtime-extractor-tls" not found Jul 10 18:41:00.238019 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:00.237922 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm podName:a220158e-f493-448c-8c20-0b3c640fc0d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:01.237912793 +0000 UTC m=+51.522464082 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8wf7f" (UID: "a220158e-f493-448c-8c20-0b3c640fc0d9") : configmap "kube-rbac-proxy" not found Jul 10 18:41:00.943666 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:00.943603 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:41:00.944049 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:00.943766 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:41:00.944049 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:00.943833 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls podName:1e89500b-d296-4aa1-afd5-359d27ba0a38 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:16.943817119 +0000 UTC m=+67.228368403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls") pod "dns-default-brwtp" (UID: "1e89500b-d296-4aa1-afd5-359d27ba0a38") : secret "dns-default-metrics-tls" not found Jul 10 18:41:01.178252 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:01.178223 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4xz7g_cb1e6736-8a0c-4dcd-bef2-a88218277f25/dns-node-resolver/0.log" Jul 10 18:41:01.247160 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:01.247086 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:01.247294 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:01.247205 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:41:01.247294 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:01.247247 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:01.247294 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:01.247259 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm podName:a220158e-f493-448c-8c20-0b3c640fc0d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:03.247243734 +0000 UTC m=+53.531795024 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8wf7f" (UID: "a220158e-f493-448c-8c20-0b3c640fc0d9") : configmap "kube-rbac-proxy" not found Jul 10 18:41:01.247400 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:01.247334 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:41:01.247400 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:01.247379 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls podName:a220158e-f493-448c-8c20-0b3c640fc0d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:03.247365678 +0000 UTC m=+53.531916982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8wf7f" (UID: "a220158e-f493-448c-8c20-0b3c640fc0d9") : secret "insights-runtime-extractor-tls" not found Jul 10 18:41:01.752388 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:01.752352 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:41:01.752545 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:01.752509 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:41:01.752545 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:01.752530 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d5f554d8-jf4wt: secret "image-registry-tls" not found Jul 10 18:41:01.752619 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:01.752598 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls podName:baf2fb14-f51e-43cc-a8af-a5fc371113c8 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:05.752582042 +0000 UTC m=+56.037133326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls") pod "image-registry-69d5f554d8-jf4wt" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8") : secret "image-registry-tls" not found Jul 10 18:41:02.576386 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:02.576361 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xksn6_988b88a2-e62e-4676-91e4-01157f069202/node-ca/0.log" Jul 10 18:41:02.660966 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:02.660926 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-vvdl8\" (UID: \"0be8606b-b59d-4412-b7fe-7f994e872ef4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:41:02.661140 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:02.661076 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:41:02.661204 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:02.661149 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls podName:0be8606b-b59d-4412-b7fe-7f994e872ef4 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:10.661129723 +0000 UTC m=+60.945681029 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls") pod "cluster-samples-operator-65db858894-vvdl8" (UID: "0be8606b-b59d-4412-b7fe-7f994e872ef4") : secret "samples-operator-tls" not found Jul 10 18:41:02.976798 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:02.976771 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w9rnj_9b74d98f-b159-4df8-9a21-79981099cc0e/serve-healthcheck-canary/0.log" Jul 10 18:41:03.266534 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.266459 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:03.266689 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.266531 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:03.266689 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:03.266570 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:41:03.266689 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:03.266635 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls podName:a220158e-f493-448c-8c20-0b3c640fc0d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:07.266616027 +0000 UTC m=+57.551167337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8wf7f" (UID: "a220158e-f493-448c-8c20-0b3c640fc0d9") : secret "insights-runtime-extractor-tls" not found Jul 10 18:41:03.266689 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:03.266664 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:41:03.266829 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:03.266739 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm podName:a220158e-f493-448c-8c20-0b3c640fc0d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:07.266723338 +0000 UTC m=+57.551274623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8wf7f" (UID: "a220158e-f493-448c-8c20-0b3c640fc0d9") : configmap "kube-rbac-proxy" not found Jul 10 18:41:03.376089 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.376057 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:03.406880 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.406855 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk"] Jul 10 18:41:03.425197 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.425177 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk"] Jul 10 18:41:03.425306 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.425273 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk" Jul 10 18:41:03.427563 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.427547 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 10 18:41:03.428462 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.428446 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-bd958\"" Jul 10 18:41:03.428510 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.428464 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 10 18:41:03.468163 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.468129 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6t9p\" (UniqueName: \"kubernetes.io/projected/bb1f8861-ccd7-4b12-bf84-e4722f9349ab-kube-api-access-h6t9p\") pod \"migrator-7f59b54f6c-pbwvk\" (UID: \"bb1f8861-ccd7-4b12-bf84-e4722f9349ab\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk" Jul 10 18:41:03.470103 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.470081 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:03.568628 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.568571 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6t9p\" (UniqueName: \"kubernetes.io/projected/bb1f8861-ccd7-4b12-bf84-e4722f9349ab-kube-api-access-h6t9p\") pod \"migrator-7f59b54f6c-pbwvk\" (UID: \"bb1f8861-ccd7-4b12-bf84-e4722f9349ab\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk" Jul 10 18:41:03.577905 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.577881 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:03.580679 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.580642 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6t9p\" (UniqueName: \"kubernetes.io/projected/bb1f8861-ccd7-4b12-bf84-e4722f9349ab-kube-api-access-h6t9p\") pod \"migrator-7f59b54f6c-pbwvk\" (UID: \"bb1f8861-ccd7-4b12-bf84-e4722f9349ab\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk" Jul 10 18:41:03.733987 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.733967 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk" Jul 10 18:41:03.872248 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:03.872224 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk"] Jul 10 18:41:04.485362 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:04.485329 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:04.568159 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:04.568125 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk" event={"ID":"bb1f8861-ccd7-4b12-bf84-e4722f9349ab","Type":"ContainerStarted","Data":"3684a493d9b56997c21ce2bf2d6e68006f05d793ee87dc8583aa905d68eb6394"} Jul 10 18:41:04.715270 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:04.715239 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:04.794173 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:04.794086 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:05.195350 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:05.195320 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:05.295479 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:05.295450 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:05.572291 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:05.572261 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk" event={"ID":"bb1f8861-ccd7-4b12-bf84-e4722f9349ab","Type":"ContainerStarted","Data":"b590563da42b241ffea2bdef8c6dde809fc529220ffb70fea4791ea2bafe9aba"} Jul 10 18:41:05.572415 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:05.572333 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk" event={"ID":"bb1f8861-ccd7-4b12-bf84-e4722f9349ab","Type":"ContainerStarted","Data":"2f473c70fbdd2cbb4674a426dde50ef2725ce8896698d141fdb0cae70317234c"} Jul 10 18:41:05.592302 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:05.592255 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pbwvk" podStartSLOduration=1.214431224 podStartE2EDuration="2.592243191s" totalImagesPullingTime="1.377811967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:05.590803191 +0000 UTC m=+55.875354498" watchObservedRunningTime="2026-07-10 18:41:05.592243191 +0000 UTC m=+55.876794519" Jul 10 18:41:05.711041 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:05.710963 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:05.788050 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:05.788024 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:05.788938 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:05.788911 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:41:05.789066 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:05.789053 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:41:05.789116 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:05.789067 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69d5f554d8-jf4wt: secret "image-registry-tls" not found Jul 10 18:41:05.789116 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:05.789113 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls podName:baf2fb14-f51e-43cc-a8af-a5fc371113c8 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:13.789098571 +0000 UTC m=+64.073649871 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls") pod "image-registry-69d5f554d8-jf4wt" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8") : secret "image-registry-tls" not found Jul 10 18:41:06.816628 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:06.816597 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:06.881895 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:06.881864 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:07.301740 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:07.301706 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:07.301924 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:07.301796 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:07.301924 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:07.301828 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:41:07.301924 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:07.301879 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:41:07.301924 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:07.301898 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm podName:a220158e-f493-448c-8c20-0b3c640fc0d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:15.301882826 +0000 UTC m=+65.586434111 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8wf7f" (UID: "a220158e-f493-448c-8c20-0b3c640fc0d9") : configmap "kube-rbac-proxy" not found Jul 10 18:41:07.301924 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:07.301918 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls podName:a220158e-f493-448c-8c20-0b3c640fc0d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:15.301907091 +0000 UTC m=+65.586458376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8wf7f" (UID: "a220158e-f493-448c-8c20-0b3c640fc0d9") : secret "insights-runtime-extractor-tls" not found Jul 10 18:41:07.829534 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:07.829503 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:08.401189 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:08.401160 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:08.588915 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:08.588892 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:08.704568 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:08.704506 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:08.769037 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:08.769019 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:09.112778 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:09.112728 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:10.089822 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:10.089793 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:10.490739 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:10.490697 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:10.499686 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:10.499662 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf64p" Jul 10 18:41:10.693532 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:10.693506 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:10.723559 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:10.723534 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-vvdl8\" (UID: \"0be8606b-b59d-4412-b7fe-7f994e872ef4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:41:10.725787 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:10.725766 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be8606b-b59d-4412-b7fe-7f994e872ef4-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-vvdl8\" (UID: \"0be8606b-b59d-4412-b7fe-7f994e872ef4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:41:10.798864 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:10.798816 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-52hbf\"" Jul 10 18:41:10.806949 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:10.806935 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" Jul 10 18:41:10.896298 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:10.896195 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:10.928273 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:10.928246 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8"] Jul 10 18:41:10.957532 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:10.957506 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:11.093595 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:11.093532 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:11.587273 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:11.587241 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" event={"ID":"0be8606b-b59d-4412-b7fe-7f994e872ef4","Type":"ContainerStarted","Data":"79cdf79e1af310b17ea733fd116ea0a88e2d7da5020eb171207fbe8028652f40"} Jul 10 18:41:12.108158 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:12.108115 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:12.388136 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:12.388100 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:12.485025 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:12.484995 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:12.584783 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:12.584754 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:12.594815 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:12.594785 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" event={"ID":"0be8606b-b59d-4412-b7fe-7f994e872ef4","Type":"ContainerStarted","Data":"a57ba8418859b8b375d0ab642dd94139100e7eb5b87dfada44a6d7c8c6965cce"} Jul 10 18:41:12.595181 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:12.594868 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" event={"ID":"0be8606b-b59d-4412-b7fe-7f994e872ef4","Type":"ContainerStarted","Data":"29e7d24a889e63d6ffd34c05b3a9b14c8abb3f7b5de720689aa21404621c74ce"} Jul 10 18:41:12.628004 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:12.627962 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-vvdl8" podStartSLOduration=17.183023693 podStartE2EDuration="18.627948484s" totalImagesPullingTime="1.444924791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:12.626249812 +0000 UTC m=+62.910801134" watchObservedRunningTime="2026-07-10 18:41:12.627948484 +0000 UTC m=+62.912499790" Jul 10 18:41:12.703252 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:12.703197 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:12.786313 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:12.786286 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:12.885373 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:12.885352 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:13.087204 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:13.087102 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:13.848847 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:13.848808 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:41:13.851080 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:13.851060 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls\") pod \"image-registry-69d5f554d8-jf4wt\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:41:13.892990 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:13.892967 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tn8fv\"" Jul 10 18:41:13.900790 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:13.900773 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:41:14.025656 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:14.025606 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69d5f554d8-jf4wt"] Jul 10 18:41:14.029374 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:14.029346 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaf2fb14_f51e_43cc_a8af_a5fc371113c8.slice/crio-c3ccbdb2e28e0e2ecd2b6d3e6ee51951c1831704fdd7d6469db129b964215e5f WatchSource:0}: Error finding container c3ccbdb2e28e0e2ecd2b6d3e6ee51951c1831704fdd7d6469db129b964215e5f: Status 404 returned error can't find the container with id c3ccbdb2e28e0e2ecd2b6d3e6ee51951c1831704fdd7d6469db129b964215e5f Jul 10 18:41:14.095413 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:14.095388 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:14.286983 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:14.286944 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:14.602730 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:14.602632 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" event={"ID":"baf2fb14-f51e-43cc-a8af-a5fc371113c8","Type":"ContainerStarted","Data":"5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a"} Jul 10 18:41:14.602730 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:14.602706 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" event={"ID":"baf2fb14-f51e-43cc-a8af-a5fc371113c8","Type":"ContainerStarted","Data":"c3ccbdb2e28e0e2ecd2b6d3e6ee51951c1831704fdd7d6469db129b964215e5f"} Jul 10 18:41:14.602908 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:14.602783 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:41:14.627784 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:14.627739 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" podStartSLOduration=17.627724917 podStartE2EDuration="17.627724917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:14.626670109 +0000 UTC m=+64.911221431" watchObservedRunningTime="2026-07-10 18:41:14.627724917 +0000 UTC m=+64.912276223" Jul 10 18:41:14.693762 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:14.693734 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:14.827384 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:14.827360 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:14.928948 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:14.928915 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:15.292314 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:15.292236 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:15.362034 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:15.362000 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:15.362172 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:15.362109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:15.362271 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:15.362255 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:41:15.362336 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:15.362330 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls podName:a220158e-f493-448c-8c20-0b3c640fc0d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:31.362309092 +0000 UTC m=+81.646860377 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8wf7f" (UID: "a220158e-f493-448c-8c20-0b3c640fc0d9") : secret "insights-runtime-extractor-tls" not found Jul 10 18:41:15.362513 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:15.362494 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a220158e-f493-448c-8c20-0b3c640fc0d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:15.966389 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:15.966353 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:41:15.969027 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:15.969011 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:41:15.979337 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:15.979313 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b22d6861-0651-43b0-8642-f1bb3a2c54ee-metrics-certs\") pod \"network-metrics-daemon-ccjtn\" (UID: \"b22d6861-0651-43b0-8642-f1bb3a2c54ee\") " pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:41:16.067142 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.067108 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:41:16.070541 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.070527 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:41:16.079754 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.079735 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1-original-pull-secret\") pod \"global-pull-secret-syncer-dj7wf\" (UID: \"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1\") " pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:41:16.112560 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.112518 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:16.168423 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.168395 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr2mx\" (UniqueName: \"kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx\") pod \"network-check-target-prjp9\" (UID: \"b691ab8e-52aa-40ea-9e2b-ce8d242181ef\") " pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:41:16.171212 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.171198 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 18:41:16.181676 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.181662 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 18:41:16.191766 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.191748 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr2mx\" (UniqueName: \"kubernetes.io/projected/b691ab8e-52aa-40ea-9e2b-ce8d242181ef-kube-api-access-gr2mx\") pod \"network-check-target-prjp9\" (UID: \"b691ab8e-52aa-40ea-9e2b-ce8d242181ef\") " pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:41:16.257540 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.257489 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-75lnr\"" Jul 10 18:41:16.261689 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.261669 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dj7wf" Jul 10 18:41:16.265435 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.265420 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjtn" Jul 10 18:41:16.270320 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.270305 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-crdnw\"" Jul 10 18:41:16.278628 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.278601 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:41:16.397818 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.397793 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dj7wf"] Jul 10 18:41:16.401319 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:16.401289 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec2f7adf_fe91_4da9_b03e_b33b6f2e4fd1.slice/crio-339b9f3e7f2df9937d0f4ea765a7e804d7d95449eab379d3f3daac876e375f0f WatchSource:0}: Error finding container 339b9f3e7f2df9937d0f4ea765a7e804d7d95449eab379d3f3daac876e375f0f: Status 404 returned error can't find the container with id 339b9f3e7f2df9937d0f4ea765a7e804d7d95449eab379d3f3daac876e375f0f Jul 10 18:41:16.413409 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.413313 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ccjtn"] Jul 10 18:41:16.415618 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:16.415592 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22d6861_0651_43b0_8642_f1bb3a2c54ee.slice/crio-28b2c418fe867b0bec4ce15a199fd486ce15099e1c4ba70dff79ce4cc3a8e482 WatchSource:0}: Error finding container 28b2c418fe867b0bec4ce15a199fd486ce15099e1c4ba70dff79ce4cc3a8e482: Status 404 returned error can't find the container with id 28b2c418fe867b0bec4ce15a199fd486ce15099e1c4ba70dff79ce4cc3a8e482 Jul 10 18:41:16.428313 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.428289 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-prjp9"] Jul 10 18:41:16.430516 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:16.430492 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb691ab8e_52aa_40ea_9e2b_ce8d242181ef.slice/crio-35200858196d5cc8c65c182683359656b82c045f2589f7b3dea5edadf87b2f55 WatchSource:0}: Error finding container 35200858196d5cc8c65c182683359656b82c045f2589f7b3dea5edadf87b2f55: Status 404 returned error can't find the container with id 35200858196d5cc8c65c182683359656b82c045f2589f7b3dea5edadf87b2f55 Jul 10 18:41:16.494862 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.494818 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:16.609789 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.609721 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ccjtn" event={"ID":"b22d6861-0651-43b0-8642-f1bb3a2c54ee","Type":"ContainerStarted","Data":"28b2c418fe867b0bec4ce15a199fd486ce15099e1c4ba70dff79ce4cc3a8e482"} Jul 10 18:41:16.610836 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.610811 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dj7wf" event={"ID":"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1","Type":"ContainerStarted","Data":"339b9f3e7f2df9937d0f4ea765a7e804d7d95449eab379d3f3daac876e375f0f"} Jul 10 18:41:16.611783 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.611764 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-prjp9" event={"ID":"b691ab8e-52aa-40ea-9e2b-ce8d242181ef","Type":"ContainerStarted","Data":"35200858196d5cc8c65c182683359656b82c045f2589f7b3dea5edadf87b2f55"} Jul 10 18:41:16.684909 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.684881 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:16.880055 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.880012 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:16.974291 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.974269 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:41:16.976550 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:16.976531 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e89500b-d296-4aa1-afd5-359d27ba0a38-metrics-tls\") pod \"dns-default-brwtp\" (UID: \"1e89500b-d296-4aa1-afd5-359d27ba0a38\") " pod="openshift-dns/dns-default-brwtp" Jul 10 18:41:17.091281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:17.091186 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:17.251446 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:17.251379 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s4tdx\"" Jul 10 18:41:17.259377 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:17.259328 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-brwtp" Jul 10 18:41:17.421069 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:17.421007 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-brwtp"] Jul 10 18:41:17.644588 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:17.644550 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e89500b_d296_4aa1_afd5_359d27ba0a38.slice/crio-7f2eec531e91c86f0ddb7b32bb5d654888208cad508659eb79a698333da17eb6 WatchSource:0}: Error finding container 7f2eec531e91c86f0ddb7b32bb5d654888208cad508659eb79a698333da17eb6: Status 404 returned error can't find the container with id 7f2eec531e91c86f0ddb7b32bb5d654888208cad508659eb79a698333da17eb6 Jul 10 18:41:18.076007 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.075842 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:18.211062 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.211011 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:18.312993 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.312916 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:18.412171 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.412109 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:18.563994 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.563905 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:18.624500 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.624467 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ccjtn" event={"ID":"b22d6861-0651-43b0-8642-f1bb3a2c54ee","Type":"ContainerStarted","Data":"6ca45fac881506b6453840f43d912bc1e0346abe21f91717d8d4808c7e8b9499"} Jul 10 18:41:18.624699 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.624530 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ccjtn" event={"ID":"b22d6861-0651-43b0-8642-f1bb3a2c54ee","Type":"ContainerStarted","Data":"9ff309f59cce3205c9eb001c92a967bcf0485fbdfe4bb9c39985833c5152c94e"} Jul 10 18:41:18.626675 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.626630 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brwtp" event={"ID":"1e89500b-d296-4aa1-afd5-359d27ba0a38","Type":"ContainerStarted","Data":"7f2eec531e91c86f0ddb7b32bb5d654888208cad508659eb79a698333da17eb6"} Jul 10 18:41:18.637779 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.637750 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:18.643537 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.643423 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ccjtn" podStartSLOduration=67.389921935 podStartE2EDuration="1m8.643406776s" totalImagesPullingTime="1.253484841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:18.642818883 +0000 UTC m=+68.927370217" watchObservedRunningTime="2026-07-10 18:41:18.643406776 +0000 UTC m=+68.927958085" Jul 10 18:41:18.704877 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.704799 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:18.806835 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.806663 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:18.905537 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.905499 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:18.979465 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:18.979361 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:19.089524 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:19.089492 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:19.190083 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:19.190034 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:19.565531 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:19.565489 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:19.665312 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:19.665286 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:20.015168 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:20.015096 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:20.599298 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:20.599247 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:20.702613 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:20.702576 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:22.641897 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:22.641861 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-prjp9" event={"ID":"b691ab8e-52aa-40ea-9e2b-ce8d242181ef","Type":"ContainerStarted","Data":"b9350a4604aabd05619f1f1ac33c1cd53e69029c938d3afdd8e17a2c89e47065"} Jul 10 18:41:22.642356 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:22.641922 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:41:22.643307 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:22.643287 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dj7wf" event={"ID":"ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1","Type":"ContainerStarted","Data":"36e123ee9ddcb6c2bef10984ca6afcbeb9e893cdddf2247d09a2613e8fc65a4d"} Jul 10 18:41:22.644910 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:22.644890 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brwtp" event={"ID":"1e89500b-d296-4aa1-afd5-359d27ba0a38","Type":"ContainerStarted","Data":"fe4e8c2bec6b5da3c9eec5cc2e84371b86121c6a7952cd3c02827053c9efee9b"} Jul 10 18:41:22.645008 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:22.644931 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brwtp" event={"ID":"1e89500b-d296-4aa1-afd5-359d27ba0a38","Type":"ContainerStarted","Data":"0a8ac4eb295346b583c53910762adbdf30167b717a60196f0c8c4125c6179373"} Jul 10 18:41:22.645008 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:22.644963 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-brwtp" Jul 10 18:41:22.661217 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:22.661175 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-prjp9" podStartSLOduration=65.940622875 podStartE2EDuration="1m11.661161027s" totalImagesPullingTime="5.720538152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:22.660305835 +0000 UTC m=+72.944857158" watchObservedRunningTime="2026-07-10 18:41:22.661161027 +0000 UTC m=+72.945712333" Jul 10 18:41:22.683503 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:22.683429 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-brwtp" podStartSLOduration=33.178574206 podStartE2EDuration="37.683410193s" totalImagesPullingTime="4.504835987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:22.682732171 +0000 UTC m=+72.967283479" watchObservedRunningTime="2026-07-10 18:41:22.683410193 +0000 UTC m=+72.967961500" Jul 10 18:41:22.700031 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:22.699985 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dj7wf" podStartSLOduration=65.939180269 podStartE2EDuration="1m11.699968726s" totalImagesPullingTime="5.760788457s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:22.69936471 +0000 UTC m=+72.983916235" watchObservedRunningTime="2026-07-10 18:41:22.699968726 +0000 UTC m=+72.984520034" Jul 10 18:41:25.235088 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:25.235054 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:25.313982 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:25.313953 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:28.726188 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:28.726156 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:29.589037 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:29.589007 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:29.677399 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:29.677370 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:29.752548 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:29.752499 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:30.001550 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:30.001521 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:31.197170 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.197055 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:31.341696 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.341662 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:31.375702 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.375678 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:31.377901 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.377874 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a220158e-f493-448c-8c20-0b3c640fc0d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wf7f\" (UID: \"a220158e-f493-448c-8c20-0b3c640fc0d9\") " pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:31.572810 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.572713 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-jpj6c"] Jul 10 18:41:31.576640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.576619 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-jpj6c" Jul 10 18:41:31.578136 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.578114 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m5ckx\"" Jul 10 18:41:31.580011 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.579997 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8wf7f" Jul 10 18:41:31.580358 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:31.580331 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-86.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-137-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 10 18:41:31.580358 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.580353 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 18:41:31.580468 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:31.580331 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-8vfnb\" is forbidden: User \"system:node:ip-10-0-137-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-137-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"default-dockercfg-8vfnb\"" type="*v1.Secret" Jul 10 18:41:31.613782 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.613711 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69d5f554d8-jf4wt"] Jul 10 18:41:31.614819 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.614770 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-jpj6c"] Jul 10 18:41:31.619415 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.619389 2563 patch_prober.go:28] interesting pod/image-registry-69d5f554d8-jf4wt container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 18:41:31.619533 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.619431 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" podUID="baf2fb14-f51e-43cc-a8af-a5fc371113c8" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 18:41:31.677771 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.677739 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk2tb\" (UniqueName: \"kubernetes.io/projected/4b00067d-6ef6-44f4-96da-ff149aa4d69c-kube-api-access-jk2tb\") pod \"downloads-68c9cbb497-jpj6c\" (UID: \"4b00067d-6ef6-44f4-96da-ff149aa4d69c\") " pod="openshift-console/downloads-68c9cbb497-jpj6c" Jul 10 18:41:31.726855 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.726835 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8wf7f"] Jul 10 18:41:31.729149 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:31.729126 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda220158e_f493_448c_8c20_0b3c640fc0d9.slice/crio-fac735a6d31751484d8dc4d723edcbba4d01eaacfb69c0cf875a772aa8dcee69 WatchSource:0}: Error finding container fac735a6d31751484d8dc4d723edcbba4d01eaacfb69c0cf875a772aa8dcee69: Status 404 returned error can't find the container with id fac735a6d31751484d8dc4d723edcbba4d01eaacfb69c0cf875a772aa8dcee69 Jul 10 18:41:31.731404 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.731381 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7848468f97-fkr27"] Jul 10 18:41:31.735447 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.735430 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.764554 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.764532 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7848468f97-fkr27"] Jul 10 18:41:31.776132 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.776114 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:31.779027 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.779008 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk2tb\" (UniqueName: \"kubernetes.io/projected/4b00067d-6ef6-44f4-96da-ff149aa4d69c-kube-api-access-jk2tb\") pod \"downloads-68c9cbb497-jpj6c\" (UID: \"4b00067d-6ef6-44f4-96da-ff149aa4d69c\") " pod="openshift-console/downloads-68c9cbb497-jpj6c" Jul 10 18:41:31.871192 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.871129 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:31.879535 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.879511 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/469babdc-6a93-4535-9436-5dbb4458405e-image-registry-private-configuration\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.879615 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.879543 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/469babdc-6a93-4535-9436-5dbb4458405e-ca-trust-extracted\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.879615 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.879565 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/469babdc-6a93-4535-9436-5dbb4458405e-registry-tls\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.879615 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.879584 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/469babdc-6a93-4535-9436-5dbb4458405e-trusted-ca\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.879749 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.879637 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/469babdc-6a93-4535-9436-5dbb4458405e-registry-certificates\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.879749 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.879696 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85g29\" (UniqueName: \"kubernetes.io/projected/469babdc-6a93-4535-9436-5dbb4458405e-kube-api-access-85g29\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.879749 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.879723 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/469babdc-6a93-4535-9436-5dbb4458405e-installation-pull-secrets\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.879749 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.879746 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/469babdc-6a93-4535-9436-5dbb4458405e-bound-sa-token\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.981033 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.981003 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/469babdc-6a93-4535-9436-5dbb4458405e-image-registry-private-configuration\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.981033 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.981036 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/469babdc-6a93-4535-9436-5dbb4458405e-ca-trust-extracted\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.981232 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.981056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/469babdc-6a93-4535-9436-5dbb4458405e-registry-tls\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.981232 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.981071 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/469babdc-6a93-4535-9436-5dbb4458405e-trusted-ca\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.981232 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.981098 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/469babdc-6a93-4535-9436-5dbb4458405e-registry-certificates\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.981232 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.981134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85g29\" (UniqueName: \"kubernetes.io/projected/469babdc-6a93-4535-9436-5dbb4458405e-kube-api-access-85g29\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.981232 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.981164 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/469babdc-6a93-4535-9436-5dbb4458405e-installation-pull-secrets\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.981232 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.981192 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/469babdc-6a93-4535-9436-5dbb4458405e-bound-sa-token\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.981502 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.981457 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/469babdc-6a93-4535-9436-5dbb4458405e-ca-trust-extracted\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.982025 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.982001 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/469babdc-6a93-4535-9436-5dbb4458405e-registry-certificates\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.982133 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.982078 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/469babdc-6a93-4535-9436-5dbb4458405e-trusted-ca\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.983683 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.983638 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/469babdc-6a93-4535-9436-5dbb4458405e-registry-tls\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.983898 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.983881 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/469babdc-6a93-4535-9436-5dbb4458405e-installation-pull-secrets\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:31.984077 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:31.984057 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/469babdc-6a93-4535-9436-5dbb4458405e-image-registry-private-configuration\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:32.003571 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.003548 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:32.004746 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.004729 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85g29\" (UniqueName: \"kubernetes.io/projected/469babdc-6a93-4535-9436-5dbb4458405e-kube-api-access-85g29\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:32.005329 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.005311 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/469babdc-6a93-4535-9436-5dbb4458405e-bound-sa-token\") pod \"image-registry-7848468f97-fkr27\" (UID: \"469babdc-6a93-4535-9436-5dbb4458405e\") " pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:32.044456 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.044436 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:32.132137 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.132102 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:32.208948 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.208916 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7848468f97-fkr27"] Jul 10 18:41:32.214849 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:32.213793 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469babdc_6a93_4535_9436_5dbb4458405e.slice/crio-a535cd85e230098802354ced6a5eed82fd1590719f79067248c5ab3a71bea15b WatchSource:0}: Error finding container a535cd85e230098802354ced6a5eed82fd1590719f79067248c5ab3a71bea15b: Status 404 returned error can't find the container with id a535cd85e230098802354ced6a5eed82fd1590719f79067248c5ab3a71bea15b Jul 10 18:41:32.652154 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.651451 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-brwtp" Jul 10 18:41:32.682544 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.681925 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8wf7f" event={"ID":"a220158e-f493-448c-8c20-0b3c640fc0d9","Type":"ContainerStarted","Data":"fac735a6d31751484d8dc4d723edcbba4d01eaacfb69c0cf875a772aa8dcee69"} Jul 10 18:41:32.684693 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.683840 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7848468f97-fkr27" event={"ID":"469babdc-6a93-4535-9436-5dbb4458405e","Type":"ContainerStarted","Data":"43093eebbb78d6aec49a65bb5711b2309f5ab60f18a5250733288ff5c42d32a1"} Jul 10 18:41:32.684693 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.684591 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:32.684693 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.684620 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7848468f97-fkr27" event={"ID":"469babdc-6a93-4535-9436-5dbb4458405e","Type":"ContainerStarted","Data":"a535cd85e230098802354ced6a5eed82fd1590719f79067248c5ab3a71bea15b"} Jul 10 18:41:32.791984 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:32.791955 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8vfnb\"" Jul 10 18:41:32.792954 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:32.792935 2563 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 10 18:41:32.793067 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:32.792967 2563 projected.go:196] Error preparing data for projected volume kube-api-access-jk2tb for pod openshift-console/downloads-68c9cbb497-jpj6c: failed to sync configmap cache: timed out waiting for the condition Jul 10 18:41:32.793067 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:32.793028 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b00067d-6ef6-44f4-96da-ff149aa4d69c-kube-api-access-jk2tb podName:4b00067d-6ef6-44f4-96da-ff149aa4d69c nodeName:}" failed. No retries permitted until 2026-07-10 18:41:33.293008518 +0000 UTC m=+83.577559817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jk2tb" (UniqueName: "kubernetes.io/projected/4b00067d-6ef6-44f4-96da-ff149aa4d69c-kube-api-access-jk2tb") pod "downloads-68c9cbb497-jpj6c" (UID: "4b00067d-6ef6-44f4-96da-ff149aa4d69c") : failed to sync configmap cache: timed out waiting for the condition Jul 10 18:41:33.116477 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:33.116401 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 18:41:33.394025 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:33.393993 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk2tb\" (UniqueName: \"kubernetes.io/projected/4b00067d-6ef6-44f4-96da-ff149aa4d69c-kube-api-access-jk2tb\") pod \"downloads-68c9cbb497-jpj6c\" (UID: \"4b00067d-6ef6-44f4-96da-ff149aa4d69c\") " pod="openshift-console/downloads-68c9cbb497-jpj6c" Jul 10 18:41:33.396433 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:33.396403 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk2tb\" (UniqueName: \"kubernetes.io/projected/4b00067d-6ef6-44f4-96da-ff149aa4d69c-kube-api-access-jk2tb\") pod \"downloads-68c9cbb497-jpj6c\" (UID: \"4b00067d-6ef6-44f4-96da-ff149aa4d69c\") " pod="openshift-console/downloads-68c9cbb497-jpj6c" Jul 10 18:41:33.685323 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:33.685250 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-jpj6c" Jul 10 18:41:35.476054 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:35.476001 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7848468f97-fkr27" podStartSLOduration=4.475984619 podStartE2EDuration="4.475984619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:32.712141143 +0000 UTC m=+82.996692452" watchObservedRunningTime="2026-07-10 18:41:35.475984619 +0000 UTC m=+85.760535925" Jul 10 18:41:35.476449 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:35.476153 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-jpj6c"] Jul 10 18:41:35.478804 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:35.478780 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b00067d_6ef6_44f4_96da_ff149aa4d69c.slice/crio-e620bf768b7ef9153d3c22477f783a59087c65f0622ff52507b7a3f1e7512b07 WatchSource:0}: Error finding container e620bf768b7ef9153d3c22477f783a59087c65f0622ff52507b7a3f1e7512b07: Status 404 returned error can't find the container with id e620bf768b7ef9153d3c22477f783a59087c65f0622ff52507b7a3f1e7512b07 Jul 10 18:41:35.697110 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:35.697084 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8wf7f" event={"ID":"a220158e-f493-448c-8c20-0b3c640fc0d9","Type":"ContainerStarted","Data":"465dd1a9f418bb1c94693e2ec7879b4322ac1844deb8874daa056b4eee530276"} Jul 10 18:41:35.698134 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:35.698116 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-jpj6c" event={"ID":"4b00067d-6ef6-44f4-96da-ff149aa4d69c","Type":"ContainerStarted","Data":"e620bf768b7ef9153d3c22477f783a59087c65f0622ff52507b7a3f1e7512b07"} Jul 10 18:41:35.842503 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:35.842479 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:35.942753 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:35.942709 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:36.704162 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:36.704124 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8wf7f" event={"ID":"a220158e-f493-448c-8c20-0b3c640fc0d9","Type":"ContainerStarted","Data":"35d3fb22203c1b85cf9415cb69d60e6f855eecd3f44ffaa229ffb59cea8c4664"} Jul 10 18:41:36.861347 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:36.861314 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:36.929882 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:36.929849 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:37.797453 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.797340 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:37.834194 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.834168 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rzjjh"] Jul 10 18:41:37.839175 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.839150 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:37.841947 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.841928 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 18:41:37.842919 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.842898 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 18:41:37.843005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.842956 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 10 18:41:37.843063 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.843012 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-gq6wv\"" Jul 10 18:41:37.843063 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.843023 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 10 18:41:37.843063 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.842959 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 18:41:37.851029 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.851010 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rzjjh"] Jul 10 18:41:37.884554 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.884300 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:37.928127 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.928096 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b37b27b3-9815-4488-a231-c024429a6b2e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:37.928266 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.928133 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9dc9\" (UniqueName: \"kubernetes.io/projected/b37b27b3-9815-4488-a231-c024429a6b2e-kube-api-access-q9dc9\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:37.928266 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.928180 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b37b27b3-9815-4488-a231-c024429a6b2e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:37.928266 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.928210 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b37b27b3-9815-4488-a231-c024429a6b2e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:37.965907 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:37.965874 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:38.029446 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:38.029417 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b37b27b3-9815-4488-a231-c024429a6b2e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:38.029713 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:38.029472 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b37b27b3-9815-4488-a231-c024429a6b2e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:38.029713 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:38.029574 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b37b27b3-9815-4488-a231-c024429a6b2e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:38.029713 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:38.029606 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9dc9\" (UniqueName: \"kubernetes.io/projected/b37b27b3-9815-4488-a231-c024429a6b2e-kube-api-access-q9dc9\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:38.030292 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:38.030264 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b37b27b3-9815-4488-a231-c024429a6b2e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:38.032337 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:38.032291 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b37b27b3-9815-4488-a231-c024429a6b2e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:38.032922 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:38.032901 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b37b27b3-9815-4488-a231-c024429a6b2e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:38.038368 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:38.038344 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9dc9\" (UniqueName: \"kubernetes.io/projected/b37b27b3-9815-4488-a231-c024429a6b2e-kube-api-access-q9dc9\") pod \"prometheus-operator-64785b6675-rzjjh\" (UID: \"b37b27b3-9815-4488-a231-c024429a6b2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:38.151171 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:38.151150 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" Jul 10 18:41:38.868500 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:38.868463 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rzjjh"] Jul 10 18:41:38.869194 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:38.869168 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb37b27b3_9815_4488_a231_c024429a6b2e.slice/crio-d61bc3a1b49070c00abbe454edffdf4b0e4fdd559e291aad9b1d24790b3f8e29 WatchSource:0}: Error finding container d61bc3a1b49070c00abbe454edffdf4b0e4fdd559e291aad9b1d24790b3f8e29: Status 404 returned error can't find the container with id d61bc3a1b49070c00abbe454edffdf4b0e4fdd559e291aad9b1d24790b3f8e29 Jul 10 18:41:39.719375 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:39.719338 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8wf7f" event={"ID":"a220158e-f493-448c-8c20-0b3c640fc0d9","Type":"ContainerStarted","Data":"e212ad24a9c8b6f34cfafd83c67c36b422d5886b8821564fc0df065461469988"} Jul 10 18:41:39.720933 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:39.720907 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" event={"ID":"b37b27b3-9815-4488-a231-c024429a6b2e","Type":"ContainerStarted","Data":"d61bc3a1b49070c00abbe454edffdf4b0e4fdd559e291aad9b1d24790b3f8e29"} Jul 10 18:41:39.750211 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:39.750160 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8wf7f" podStartSLOduration=33.837818454 podStartE2EDuration="40.750139064s" totalImagesPullingTime="6.91232061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:40:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:39.749793353 +0000 UTC m=+90.034344689" watchObservedRunningTime="2026-07-10 18:41:39.750139064 +0000 UTC m=+90.034690372" Jul 10 18:41:40.317068 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:40.316981 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:40.433108 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:40.433074 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:40.533619 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:40.533584 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:40.726222 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:40.726139 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" event={"ID":"b37b27b3-9815-4488-a231-c024429a6b2e","Type":"ContainerStarted","Data":"ad1e3d5559dbac1a0eb6a515436ef075d32d1e262a8bd75757d79dd3f7491ab7"} Jul 10 18:41:40.726222 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:40.726203 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" event={"ID":"b37b27b3-9815-4488-a231-c024429a6b2e","Type":"ContainerStarted","Data":"568e860f2176f447d6e5a7037074d3c22fa322ad7900dbe67fefc46a0bceab81"} Jul 10 18:41:40.746188 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:40.746145 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-rzjjh" podStartSLOduration=2.5056678 podStartE2EDuration="3.746130506s" totalImagesPullingTime="1.240462706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:40.744915972 +0000 UTC m=+91.029467312" watchObservedRunningTime="2026-07-10 18:41:40.746130506 +0000 UTC m=+91.030681813" Jul 10 18:41:41.619181 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:41.619151 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:41:41.915007 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:41.914969 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:42.041967 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:42.041918 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:42.947271 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:42.947239 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:43.088368 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:43.088334 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:43.157712 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:43.157680 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:44.965484 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:44.965449 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:45.032987 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:45.032947 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:45.925079 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:45.925042 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:46.027040 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.027002 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:46.584245 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.584194 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:46.728241 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.728204 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:46.822597 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.822564 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rp2p5"] Jul 10 18:41:46.827269 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.827249 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:46.829807 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.829785 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 18:41:46.829922 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.829879 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 18:41:46.830017 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.830003 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 18:41:46.830594 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.830561 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6r84z\"" Jul 10 18:41:46.864564 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.864507 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:46.903898 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.903866 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/94b134de-c48b-4194-8cba-0f93ad8481d9-root\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:46.904005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.903906 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6t55\" (UniqueName: \"kubernetes.io/projected/94b134de-c48b-4194-8cba-0f93ad8481d9-kube-api-access-w6t55\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:46.904005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.903936 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:46.904005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.903965 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-textfile\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:46.904005 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.903989 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:46.904198 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.904010 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-wtmp\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:46.904198 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.904093 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-tls\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:46.904198 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.904131 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94b134de-c48b-4194-8cba-0f93ad8481d9-sys\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:46.904198 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.904162 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94b134de-c48b-4194-8cba-0f93ad8481d9-metrics-client-ca\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:46.962915 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:46.962854 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:47.004799 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.004772 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-wtmp\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.004917 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.004814 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-tls\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.004917 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.004846 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94b134de-c48b-4194-8cba-0f93ad8481d9-sys\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.004917 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.004884 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94b134de-c48b-4194-8cba-0f93ad8481d9-metrics-client-ca\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.005080 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.004917 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/94b134de-c48b-4194-8cba-0f93ad8481d9-root\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.005080 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.004940 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6t55\" (UniqueName: \"kubernetes.io/projected/94b134de-c48b-4194-8cba-0f93ad8481d9-kube-api-access-w6t55\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.005080 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.004954 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94b134de-c48b-4194-8cba-0f93ad8481d9-sys\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.005080 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.004972 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.005080 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:47.004963 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 18:41:47.005080 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.005009 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/94b134de-c48b-4194-8cba-0f93ad8481d9-root\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.005080 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.005014 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-textfile\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.005080 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.004971 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-wtmp\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.005080 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:47.005050 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-tls podName:94b134de-c48b-4194-8cba-0f93ad8481d9 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:47.505026416 +0000 UTC m=+97.789577704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-tls") pod "node-exporter-rp2p5" (UID: "94b134de-c48b-4194-8cba-0f93ad8481d9") : secret "node-exporter-tls" not found Jul 10 18:41:47.005453 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.005135 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.005504 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.005464 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-textfile\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.005591 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.005569 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.005668 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.005579 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94b134de-c48b-4194-8cba-0f93ad8481d9-metrics-client-ca\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.007713 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.007690 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.018406 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.018386 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6t55\" (UniqueName: \"kubernetes.io/projected/94b134de-c48b-4194-8cba-0f93ad8481d9-kube-api-access-w6t55\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.062336 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.062301 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:47.158549 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.158493 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:47.306997 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.306963 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:47.408432 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.408397 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:47.509249 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.509179 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-tls\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.512008 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.511954 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/94b134de-c48b-4194-8cba-0f93ad8481d9-node-exporter-tls\") pod \"node-exporter-rp2p5\" (UID: \"94b134de-c48b-4194-8cba-0f93ad8481d9\") " pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.512946 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.512918 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:47.611059 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.610985 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:47.661661 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.661623 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:41:47.666978 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.666955 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.670546 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.670491 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 18:41:47.670546 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.670533 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 18:41:47.671214 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.671093 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 18:41:47.671214 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.671119 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 18:41:47.671214 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.671132 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 18:41:47.671214 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.671167 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 18:41:47.671472 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.671407 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 18:41:47.671792 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.671775 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bddn4\"" Jul 10 18:41:47.672121 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.672096 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 18:41:47.672323 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.672307 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 18:41:47.685201 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.685173 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:41:47.715040 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.714984 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:47.740900 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.740855 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rp2p5" Jul 10 18:41:47.812695 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.812603 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.812695 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.812672 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13e879a6-6b80-4809-983b-fcca92cc36f7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.812911 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.812703 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.812911 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.812741 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.812911 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.812766 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-config-volume\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.812911 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.812852 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.812911 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.812878 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13e879a6-6b80-4809-983b-fcca92cc36f7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.812911 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.812903 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr44h\" (UniqueName: \"kubernetes.io/projected/13e879a6-6b80-4809-983b-fcca92cc36f7-kube-api-access-lr44h\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.813149 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.812970 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.813149 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.813003 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-web-config\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.813149 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.813041 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13e879a6-6b80-4809-983b-fcca92cc36f7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.813149 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.813076 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13e879a6-6b80-4809-983b-fcca92cc36f7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.813149 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.813098 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13e879a6-6b80-4809-983b-fcca92cc36f7-config-out\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.914510 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13e879a6-6b80-4809-983b-fcca92cc36f7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.914510 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914512 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13e879a6-6b80-4809-983b-fcca92cc36f7-config-out\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.914771 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914538 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.914771 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914562 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13e879a6-6b80-4809-983b-fcca92cc36f7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.914771 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914586 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.914771 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914677 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.914771 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914712 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-config-volume\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.914771 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.914771 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914766 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13e879a6-6b80-4809-983b-fcca92cc36f7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.915117 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914788 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr44h\" (UniqueName: \"kubernetes.io/projected/13e879a6-6b80-4809-983b-fcca92cc36f7-kube-api-access-lr44h\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.915117 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.915117 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914861 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-web-config\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.915117 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.914902 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13e879a6-6b80-4809-983b-fcca92cc36f7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.915117 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.915028 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13e879a6-6b80-4809-983b-fcca92cc36f7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.916245 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.915635 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13e879a6-6b80-4809-983b-fcca92cc36f7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.916245 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.915857 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13e879a6-6b80-4809-983b-fcca92cc36f7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.916245 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:47.915961 2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 10 18:41:47.916245 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:47.916023 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-main-tls podName:13e879a6-6b80-4809-983b-fcca92cc36f7 nodeName:}" failed. No retries permitted until 2026-07-10 18:41:48.416002743 +0000 UTC m=+98.700554028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "13e879a6-6b80-4809-983b-fcca92cc36f7") : secret "alertmanager-main-tls" not found Jul 10 18:41:47.917863 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.917778 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.917963 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.917907 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13e879a6-6b80-4809-983b-fcca92cc36f7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.918340 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.918318 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-web-config\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.918429 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.918326 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.918663 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.918621 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13e879a6-6b80-4809-983b-fcca92cc36f7-config-out\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.918744 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.918663 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.919150 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.919126 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.919735 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.919635 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-config-volume\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:47.926580 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:47.926561 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr44h\" (UniqueName: \"kubernetes.io/projected/13e879a6-6b80-4809-983b-fcca92cc36f7-kube-api-access-lr44h\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:48.334924 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.334847 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:48.418148 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.418116 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:48.420809 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.420784 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13e879a6-6b80-4809-983b-fcca92cc36f7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13e879a6-6b80-4809-983b-fcca92cc36f7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:48.578098 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.578069 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:41:48.629402 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.629375 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6758dff5fd-mcqql"] Jul 10 18:41:48.633231 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.633146 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.636253 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.636047 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 10 18:41:48.636253 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.636060 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 10 18:41:48.636253 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.636047 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 10 18:41:48.636253 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.636128 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 10 18:41:48.636253 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.636232 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c58vob0fj3jub\"" Jul 10 18:41:48.637274 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.637256 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 10 18:41:48.637387 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.637295 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nwchk\"" Jul 10 18:41:48.647056 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.647036 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6758dff5fd-mcqql"] Jul 10 18:41:48.720495 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.720466 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.720712 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.720505 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5zk8\" (UniqueName: \"kubernetes.io/projected/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-kube-api-access-d5zk8\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.720712 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.720537 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-metrics-client-ca\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.720712 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.720601 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-tls\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.720712 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.720626 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-grpc-tls\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.720911 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.720731 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.720911 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.720769 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.720911 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.720835 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.821362 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.821334 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.821547 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.821394 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.821547 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.821430 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5zk8\" (UniqueName: \"kubernetes.io/projected/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-kube-api-access-d5zk8\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.821547 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.821463 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-metrics-client-ca\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.821547 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.821506 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-tls\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.821547 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.821529 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-grpc-tls\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.821837 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.821570 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.821837 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.821597 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.822344 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.822283 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-metrics-client-ca\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.824538 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.824507 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.824677 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.824553 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.824749 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.824688 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.825232 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.825211 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-grpc-tls\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.825319 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.825275 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-tls\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.825564 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.825539 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.830707 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.830685 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5zk8\" (UniqueName: \"kubernetes.io/projected/7800a84d-dbef-4e3b-891f-2685ceeb5aa1-kube-api-access-d5zk8\") pod \"thanos-querier-6758dff5fd-mcqql\" (UID: \"7800a84d-dbef-4e3b-891f-2685ceeb5aa1\") " pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:48.944863 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:48.944792 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:41:49.598802 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:49.598763 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:49.671980 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:49.671945 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:49.815839 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:49.815809 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:49.893783 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:49.893749 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:50.818126 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:50.818086 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:50.921260 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:50.921225 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:51.172464 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.172431 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-68bb6996cc-xbbz9"] Jul 10 18:41:51.208136 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.208103 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-68bb6996cc-xbbz9"] Jul 10 18:41:51.208305 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.208248 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.211867 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.211844 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-92n5j9l67bjed\"" Jul 10 18:41:51.212008 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.211907 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 10 18:41:51.212008 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.211929 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 10 18:41:51.212158 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.212099 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 10 18:41:51.212224 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.212164 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5vrbv\"" Jul 10 18:41:51.212224 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.212200 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 18:41:51.292762 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.292724 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:51.305206 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.305183 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr"] Jul 10 18:41:51.322436 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.322413 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" Jul 10 18:41:51.324426 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.324379 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr"] Jul 10 18:41:51.325212 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.325190 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 10 18:41:51.325319 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.325213 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-87qn8\"" Jul 10 18:41:51.339760 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.339740 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f9967697-4322-4e0c-a0ac-5bb77a278420-client-ca-bundle\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.339919 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.339794 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f9967697-4322-4e0c-a0ac-5bb77a278420-metrics-server-audit-profiles\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.339919 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.339822 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwgt6\" (UniqueName: \"kubernetes.io/projected/f9967697-4322-4e0c-a0ac-5bb77a278420-kube-api-access-nwgt6\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.339919 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.339845 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9967697-4322-4e0c-a0ac-5bb77a278420-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.339919 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.339892 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f9967697-4322-4e0c-a0ac-5bb77a278420-secret-metrics-server-tls\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.340127 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.339975 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f9967697-4322-4e0c-a0ac-5bb77a278420-audit-log\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.340127 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.340027 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f9967697-4322-4e0c-a0ac-5bb77a278420-secret-metrics-server-client-certs\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.391055 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.390967 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:51.440724 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.440634 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f9967697-4322-4e0c-a0ac-5bb77a278420-metrics-server-audit-profiles\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.440724 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.440682 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwgt6\" (UniqueName: \"kubernetes.io/projected/f9967697-4322-4e0c-a0ac-5bb77a278420-kube-api-access-nwgt6\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.440724 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.440715 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9967697-4322-4e0c-a0ac-5bb77a278420-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.440973 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.440742 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f9967697-4322-4e0c-a0ac-5bb77a278420-secret-metrics-server-tls\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.440973 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.440785 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7cf3518b-0d8d-4bda-935a-b40a88dc0114-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-srmfr\" (UID: \"7cf3518b-0d8d-4bda-935a-b40a88dc0114\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" Jul 10 18:41:51.440973 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.440864 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f9967697-4322-4e0c-a0ac-5bb77a278420-audit-log\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.440973 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.440901 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f9967697-4322-4e0c-a0ac-5bb77a278420-secret-metrics-server-client-certs\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.441164 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.440974 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f9967697-4322-4e0c-a0ac-5bb77a278420-client-ca-bundle\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.441164 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.441007 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4275\" (UniqueName: \"kubernetes.io/projected/7cf3518b-0d8d-4bda-935a-b40a88dc0114-kube-api-access-f4275\") pod \"monitoring-plugin-5f68df7f58-srmfr\" (UID: \"7cf3518b-0d8d-4bda-935a-b40a88dc0114\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" Jul 10 18:41:51.441465 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.441414 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f9967697-4322-4e0c-a0ac-5bb77a278420-audit-log\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.441605 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.441576 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9967697-4322-4e0c-a0ac-5bb77a278420-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.441814 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.441784 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f9967697-4322-4e0c-a0ac-5bb77a278420-metrics-server-audit-profiles\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.443963 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.443935 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f9967697-4322-4e0c-a0ac-5bb77a278420-client-ca-bundle\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.444055 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.444013 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f9967697-4322-4e0c-a0ac-5bb77a278420-secret-metrics-server-client-certs\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.444545 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.444523 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f9967697-4322-4e0c-a0ac-5bb77a278420-secret-metrics-server-tls\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.449716 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.449686 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwgt6\" (UniqueName: \"kubernetes.io/projected/f9967697-4322-4e0c-a0ac-5bb77a278420-kube-api-access-nwgt6\") pod \"metrics-server-68bb6996cc-xbbz9\" (UID: \"f9967697-4322-4e0c-a0ac-5bb77a278420\") " pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.470221 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.470193 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:51.519062 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.519029 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:41:51.542445 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.542413 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4275\" (UniqueName: \"kubernetes.io/projected/7cf3518b-0d8d-4bda-935a-b40a88dc0114-kube-api-access-f4275\") pod \"monitoring-plugin-5f68df7f58-srmfr\" (UID: \"7cf3518b-0d8d-4bda-935a-b40a88dc0114\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" Jul 10 18:41:51.542604 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.542517 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7cf3518b-0d8d-4bda-935a-b40a88dc0114-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-srmfr\" (UID: \"7cf3518b-0d8d-4bda-935a-b40a88dc0114\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" Jul 10 18:41:51.545281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.545256 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7cf3518b-0d8d-4bda-935a-b40a88dc0114-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-srmfr\" (UID: \"7cf3518b-0d8d-4bda-935a-b40a88dc0114\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" Jul 10 18:41:51.551492 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.551471 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4275\" (UniqueName: \"kubernetes.io/projected/7cf3518b-0d8d-4bda-935a-b40a88dc0114-kube-api-access-f4275\") pod \"monitoring-plugin-5f68df7f58-srmfr\" (UID: \"7cf3518b-0d8d-4bda-935a-b40a88dc0114\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" Jul 10 18:41:51.635269 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:51.635230 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" Jul 10 18:41:52.910179 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:52.910090 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:53.651456 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:53.651427 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-prjp9" Jul 10 18:41:53.782356 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:53.782258 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rp2p5" event={"ID":"94b134de-c48b-4194-8cba-0f93ad8481d9","Type":"ContainerStarted","Data":"6d8bdaf96319f54dceb4974528be2168840997a83b3d449de1d3f1336ba4d531"} Jul 10 18:41:53.908706 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:53.908624 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:41:53.914898 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:53.914878 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr"] Jul 10 18:41:53.992131 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:53.992097 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13e879a6_6b80_4809_983b_fcca92cc36f7.slice/crio-ae23de7e366f57c470b57579896cae237bb310a2e98ce91aca9f37ff5156ce63 WatchSource:0}: Error finding container ae23de7e366f57c470b57579896cae237bb310a2e98ce91aca9f37ff5156ce63: Status 404 returned error can't find the container with id ae23de7e366f57c470b57579896cae237bb310a2e98ce91aca9f37ff5156ce63 Jul 10 18:41:53.992816 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:53.992796 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf3518b_0d8d_4bda_935a_b40a88dc0114.slice/crio-d3f53bf994156935bcb23c023a60567d105fa3954d5be696b988dbc7fac5ec2c WatchSource:0}: Error finding container d3f53bf994156935bcb23c023a60567d105fa3954d5be696b988dbc7fac5ec2c: Status 404 returned error can't find the container with id d3f53bf994156935bcb23c023a60567d105fa3954d5be696b988dbc7fac5ec2c Jul 10 18:41:54.141591 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.141568 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-68bb6996cc-xbbz9"] Jul 10 18:41:54.144307 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:54.144277 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9967697_4322_4e0c_a0ac_5bb77a278420.slice/crio-7b90897eacd67c9ad4a99ed258b5c27bffe85ca653e16f9a625336f26571cfc5 WatchSource:0}: Error finding container 7b90897eacd67c9ad4a99ed258b5c27bffe85ca653e16f9a625336f26571cfc5: Status 404 returned error can't find the container with id 7b90897eacd67c9ad4a99ed258b5c27bffe85ca653e16f9a625336f26571cfc5 Jul 10 18:41:54.146861 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.146638 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6758dff5fd-mcqql"] Jul 10 18:41:54.149135 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:41:54.149110 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7800a84d_dbef_4e3b_891f_2685ceeb5aa1.slice/crio-9a18ce8de79538ab947a8ba832b908c2a6353d9da50c05ed116a07bd79b5c5d8 WatchSource:0}: Error finding container 9a18ce8de79538ab947a8ba832b908c2a6353d9da50c05ed116a07bd79b5c5d8: Status 404 returned error can't find the container with id 9a18ce8de79538ab947a8ba832b908c2a6353d9da50c05ed116a07bd79b5c5d8 Jul 10 18:41:54.699801 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.699705 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7848468f97-fkr27" Jul 10 18:41:54.790423 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.790389 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" event={"ID":"7800a84d-dbef-4e3b-891f-2685ceeb5aa1","Type":"ContainerStarted","Data":"9a18ce8de79538ab947a8ba832b908c2a6353d9da50c05ed116a07bd79b5c5d8"} Jul 10 18:41:54.794921 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.794811 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e879a6-6b80-4809-983b-fcca92cc36f7","Type":"ContainerStarted","Data":"ae23de7e366f57c470b57579896cae237bb310a2e98ce91aca9f37ff5156ce63"} Jul 10 18:41:54.796605 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.796540 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" event={"ID":"f9967697-4322-4e0c-a0ac-5bb77a278420","Type":"ContainerStarted","Data":"7b90897eacd67c9ad4a99ed258b5c27bffe85ca653e16f9a625336f26571cfc5"} Jul 10 18:41:54.799297 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.798587 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-jpj6c" event={"ID":"4b00067d-6ef6-44f4-96da-ff149aa4d69c","Type":"ContainerStarted","Data":"eb2b13cfdb53f5a6c3d479582391941b7ff33c819188f11c74bc48691d01db47"} Jul 10 18:41:54.799297 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.798640 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-jpj6c" Jul 10 18:41:54.801527 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.801501 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" event={"ID":"7cf3518b-0d8d-4bda-935a-b40a88dc0114","Type":"ContainerStarted","Data":"d3f53bf994156935bcb23c023a60567d105fa3954d5be696b988dbc7fac5ec2c"} Jul 10 18:41:54.810249 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.809563 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-jpj6c" Jul 10 18:41:54.818494 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.818133 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-jpj6c" podStartSLOduration=5.247458872 podStartE2EDuration="23.818113866s" totalImagesPullingTime="18.570654994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:54.816772435 +0000 UTC m=+105.101323743" watchObservedRunningTime="2026-07-10 18:41:54.818113866 +0000 UTC m=+105.102665175" Jul 10 18:41:54.868163 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.868100 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:54.969829 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:54.969612 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:55.070861 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:55.070806 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:55.810666 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:55.810224 2563 generic.go:356] "Generic (PLEG): container finished" podID="94b134de-c48b-4194-8cba-0f93ad8481d9" containerID="14747bdcb3bade4184ebe8ad775a08d7383e43a591de0ec1a71e0c8ad4afa4bc" exitCode=0 Jul 10 18:41:55.811866 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:55.811721 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rp2p5" event={"ID":"94b134de-c48b-4194-8cba-0f93ad8481d9","Type":"ContainerDied","Data":"14747bdcb3bade4184ebe8ad775a08d7383e43a591de0ec1a71e0c8ad4afa4bc"} Jul 10 18:41:56.638804 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:56.638749 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" podUID="baf2fb14-f51e-43cc-a8af-a5fc371113c8" containerName="registry" containerID="cri-o://5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a" gracePeriod=30 Jul 10 18:41:56.828360 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:56.828262 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:56.921175 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:56.921089 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:57.072616 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:57.072534 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:57.160035 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:57.159956 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:57.529466 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:57.529431 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:57.633109 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:57.633021 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:58.213118 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.213009 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:58.329582 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.329546 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:41:58.551760 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.551668 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:41:58.717313 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.717275 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baf2fb14-f51e-43cc-a8af-a5fc371113c8-ca-trust-extracted\") pod \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " Jul 10 18:41:58.717492 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.717358 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-trusted-ca\") pod \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " Jul 10 18:41:58.717492 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.717401 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-certificates\") pod \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " Jul 10 18:41:58.717492 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.717442 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-bound-sa-token\") pod \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " Jul 10 18:41:58.718281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.717765 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-installation-pull-secrets\") pod \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " Jul 10 18:41:58.718281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.717801 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "baf2fb14-f51e-43cc-a8af-a5fc371113c8" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:41:58.718281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.717810 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls\") pod \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " Jul 10 18:41:58.718281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.717927 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pjhp\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-kube-api-access-4pjhp\") pod \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " Jul 10 18:41:58.718281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.717967 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-image-registry-private-configuration\") pod \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\" (UID: \"baf2fb14-f51e-43cc-a8af-a5fc371113c8\") " Jul 10 18:41:58.718281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.717996 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "baf2fb14-f51e-43cc-a8af-a5fc371113c8" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:41:58.718281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.718200 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-trusted-ca\") on node \"ip-10-0-137-86.ec2.internal\" DevicePath \"\"" Jul 10 18:41:58.718281 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.718221 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-certificates\") on node \"ip-10-0-137-86.ec2.internal\" DevicePath \"\"" Jul 10 18:41:58.721438 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.721407 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "baf2fb14-f51e-43cc-a8af-a5fc371113c8" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:41:58.721551 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.721452 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "baf2fb14-f51e-43cc-a8af-a5fc371113c8" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:41:58.722063 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.722027 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-kube-api-access-4pjhp" (OuterVolumeSpecName: "kube-api-access-4pjhp") pod "baf2fb14-f51e-43cc-a8af-a5fc371113c8" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8"). InnerVolumeSpecName "kube-api-access-4pjhp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:41:58.722433 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.722396 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "baf2fb14-f51e-43cc-a8af-a5fc371113c8" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:41:58.723182 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.723156 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "baf2fb14-f51e-43cc-a8af-a5fc371113c8" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:41:58.729071 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.729042 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baf2fb14-f51e-43cc-a8af-a5fc371113c8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "baf2fb14-f51e-43cc-a8af-a5fc371113c8" (UID: "baf2fb14-f51e-43cc-a8af-a5fc371113c8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:41:58.818632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.818604 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-bound-sa-token\") on node \"ip-10-0-137-86.ec2.internal\" DevicePath \"\"" Jul 10 18:41:58.818632 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.818633 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-installation-pull-secrets\") on node \"ip-10-0-137-86.ec2.internal\" DevicePath \"\"" Jul 10 18:41:58.818842 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.818659 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-registry-tls\") on node \"ip-10-0-137-86.ec2.internal\" DevicePath \"\"" Jul 10 18:41:58.818842 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.818672 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pjhp\" (UniqueName: \"kubernetes.io/projected/baf2fb14-f51e-43cc-a8af-a5fc371113c8-kube-api-access-4pjhp\") on node \"ip-10-0-137-86.ec2.internal\" DevicePath \"\"" Jul 10 18:41:58.818842 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.818682 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baf2fb14-f51e-43cc-a8af-a5fc371113c8-image-registry-private-configuration\") on node \"ip-10-0-137-86.ec2.internal\" DevicePath \"\"" Jul 10 18:41:58.818842 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.818694 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baf2fb14-f51e-43cc-a8af-a5fc371113c8-ca-trust-extracted\") on node \"ip-10-0-137-86.ec2.internal\" DevicePath \"\"" Jul 10 18:41:58.829852 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.829816 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rp2p5" event={"ID":"94b134de-c48b-4194-8cba-0f93ad8481d9","Type":"ContainerStarted","Data":"0ccd5bbec7383a52daef3fbd4e358b72f74f78633a80da7e9a271bf26387b5c6"} Jul 10 18:41:58.832441 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.831616 2563 generic.go:356] "Generic (PLEG): container finished" podID="baf2fb14-f51e-43cc-a8af-a5fc371113c8" containerID="5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a" exitCode=0 Jul 10 18:41:58.832441 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.831721 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" event={"ID":"baf2fb14-f51e-43cc-a8af-a5fc371113c8","Type":"ContainerDied","Data":"5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a"} Jul 10 18:41:58.832441 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.831766 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" event={"ID":"baf2fb14-f51e-43cc-a8af-a5fc371113c8","Type":"ContainerDied","Data":"c3ccbdb2e28e0e2ecd2b6d3e6ee51951c1831704fdd7d6469db129b964215e5f"} Jul 10 18:41:58.832441 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.831795 2563 scope.go:117] "RemoveContainer" containerID="5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a" Jul 10 18:41:58.832441 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.831967 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69d5f554d8-jf4wt" Jul 10 18:41:58.837859 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.837829 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" event={"ID":"7cf3518b-0d8d-4bda-935a-b40a88dc0114","Type":"ContainerStarted","Data":"2e9ac0912041a33bef36baeacc18620b25328c3b24cf55d92aa92b0849645f10"} Jul 10 18:41:58.838936 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.838721 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" Jul 10 18:41:58.840852 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.840545 2563 generic.go:356] "Generic (PLEG): container finished" podID="13e879a6-6b80-4809-983b-fcca92cc36f7" containerID="c8829c4441877f4fee667bb0b63653cd5a6c04a8d481a4d0b1b6c020681dc9c1" exitCode=0 Jul 10 18:41:58.840852 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.840626 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e879a6-6b80-4809-983b-fcca92cc36f7","Type":"ContainerDied","Data":"c8829c4441877f4fee667bb0b63653cd5a6c04a8d481a4d0b1b6c020681dc9c1"} Jul 10 18:41:58.843998 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.843950 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" event={"ID":"f9967697-4322-4e0c-a0ac-5bb77a278420","Type":"ContainerStarted","Data":"d8cc763077fcb70413005337869efd38293b29e72ff18adea83872cd3cdda230"} Jul 10 18:41:58.845943 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.845926 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" Jul 10 18:41:58.850663 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.850376 2563 scope.go:117] "RemoveContainer" containerID="5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a" Jul 10 18:41:58.850863 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:41:58.850834 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a\": container with ID starting with 5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a not found: ID does not exist" containerID="5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a" Jul 10 18:41:58.850922 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.850866 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a"} err="failed to get container status \"5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a\": rpc error: code = NotFound desc = could not find container \"5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a\": container with ID starting with 5c82fab1cd460bfcb903b0895bfc088d45c04cf063cc4999dbdf23ee680c586a not found: ID does not exist" Jul 10 18:41:58.861013 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.860914 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-srmfr" podStartSLOduration=3.460308726 podStartE2EDuration="7.86089294s" totalImagesPullingTime="4.400584214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:58.859171165 +0000 UTC m=+109.143722702" watchObservedRunningTime="2026-07-10 18:41:58.86089294 +0000 UTC m=+109.145444227" Jul 10 18:41:58.932712 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:58.931898 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" podStartSLOduration=3.669127564 podStartE2EDuration="7.931878977s" totalImagesPullingTime="4.262751413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:58.930836706 +0000 UTC m=+109.215388048" watchObservedRunningTime="2026-07-10 18:41:58.931878977 +0000 UTC m=+109.216430284" Jul 10 18:41:59.051472 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:59.051410 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69d5f554d8-jf4wt"] Jul 10 18:41:59.056438 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:59.056409 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69d5f554d8-jf4wt"] Jul 10 18:41:59.852884 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:59.852845 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rp2p5" event={"ID":"94b134de-c48b-4194-8cba-0f93ad8481d9","Type":"ContainerStarted","Data":"aea302a451be08ab8b47e2ea175edb69d0e66ebeb622c5286512f3eebc76544c"} Jul 10 18:41:59.856894 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:59.856814 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" event={"ID":"7800a84d-dbef-4e3b-891f-2685ceeb5aa1","Type":"ContainerStarted","Data":"afcd8917de4914a160efd8c31ff5283daba3cc7f63b99b60a60206f62a50ecc9"} Jul 10 18:41:59.857039 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:59.856946 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" event={"ID":"7800a84d-dbef-4e3b-891f-2685ceeb5aa1","Type":"ContainerStarted","Data":"e20120e70c983ee956542dbe829c863f8cea76076262888e4b3bed233cd83f2e"} Jul 10 18:41:59.857039 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:41:59.856993 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" event={"ID":"7800a84d-dbef-4e3b-891f-2685ceeb5aa1","Type":"ContainerStarted","Data":"9d94d4f6b8a5430345ebe3f85ff9abe592532ab1340e1eb5865f70bc0dfe9d40"} Jul 10 18:42:00.341262 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:00.341231 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baf2fb14-f51e-43cc-a8af-a5fc371113c8" path="/var/lib/kubelet/pods/baf2fb14-f51e-43cc-a8af-a5fc371113c8/volumes" Jul 10 18:42:01.869541 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.869446 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" event={"ID":"7800a84d-dbef-4e3b-891f-2685ceeb5aa1","Type":"ContainerStarted","Data":"e54c0422352bb6fdbe769d5018b10dcbbec452a31de779b9fd9e311b19bf1166"} Jul 10 18:42:01.869995 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.869573 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" event={"ID":"7800a84d-dbef-4e3b-891f-2685ceeb5aa1","Type":"ContainerStarted","Data":"52c165f9fc6779d50d0c7d6d07f52dad539f21beb693487ac8bd47e1251928fc"} Jul 10 18:42:01.869995 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.869626 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:42:01.869995 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.869691 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" event={"ID":"7800a84d-dbef-4e3b-891f-2685ceeb5aa1","Type":"ContainerStarted","Data":"b4e20ddde92c82caf25b44b5d96e1da8251590f1c2cfc8c8d364f188e6813689"} Jul 10 18:42:01.873280 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.873250 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e879a6-6b80-4809-983b-fcca92cc36f7","Type":"ContainerStarted","Data":"4293b6e13f6460778af8d403292c150a6d07a571baba2a6f72276795f1938af6"} Jul 10 18:42:01.873401 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.873308 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e879a6-6b80-4809-983b-fcca92cc36f7","Type":"ContainerStarted","Data":"45c121149fde4dede85f6b243c604c2b93e8a1200e49da10ac246f3031394796"} Jul 10 18:42:01.873401 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.873341 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e879a6-6b80-4809-983b-fcca92cc36f7","Type":"ContainerStarted","Data":"2f0a670d92c7c68e265398bad203a9bfdb9ebff2774612814ca031182c97d551"} Jul 10 18:42:01.873401 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.873387 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e879a6-6b80-4809-983b-fcca92cc36f7","Type":"ContainerStarted","Data":"9ac8a93039989842138ee60183702a5a58bb6c2d8cdc098c16e1524bcf11e4da"} Jul 10 18:42:01.873537 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.873414 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e879a6-6b80-4809-983b-fcca92cc36f7","Type":"ContainerStarted","Data":"d5054ff6cf434cf06afcaf97131afcb9fc77c7bc9631b5c9cca5267540ec91bc"} Jul 10 18:42:01.873537 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.873436 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e879a6-6b80-4809-983b-fcca92cc36f7","Type":"ContainerStarted","Data":"2196d8033e660b00380cd80bf11bab5e98e73877f9a774a27933e9525aa6ba89"} Jul 10 18:42:01.898501 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.897692 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rp2p5" podStartSLOduration=14.739078363 podStartE2EDuration="15.897673667s" totalImagesPullingTime="1.014428599s" totalInitContainerRuntime="144.166705ms" isStatefulPod=true podCreationTimestamp="2026-07-10 18:41:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:41:59.883085131 +0000 UTC m=+110.167636463" watchObservedRunningTime="2026-07-10 18:42:01.897673667 +0000 UTC m=+112.182224977" Jul 10 18:42:01.898501 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.897838 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" podStartSLOduration=7.416679628 podStartE2EDuration="13.897828683s" totalImagesPullingTime="6.481149055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:42:01.896897356 +0000 UTC m=+112.181448701" watchObservedRunningTime="2026-07-10 18:42:01.897828683 +0000 UTC m=+112.182379990" Jul 10 18:42:01.939122 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:01.939084 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=8.412825109 podStartE2EDuration="14.939072139s" totalImagesPullingTime="6.504076109s" totalInitContainerRuntime="22.170921ms" isStatefulPod=false podCreationTimestamp="2026-07-10 18:41:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:42:01.936864925 +0000 UTC m=+112.221416233" watchObservedRunningTime="2026-07-10 18:42:01.939072139 +0000 UTC m=+112.223623446" Jul 10 18:42:04.758741 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:04.758715 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w9rnj_9b74d98f-b159-4df8-9a21-79981099cc0e/serve-healthcheck-canary/0.log" Jul 10 18:42:06.651435 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:06.651393 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:07.883946 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:07.883920 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6758dff5fd-mcqql" Jul 10 18:42:08.200040 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:08.199966 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:08.259799 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:08.259773 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:10.224935 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:10.224904 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:10.325982 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:10.325956 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:11.369159 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:11.369134 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:11.471485 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:11.471462 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:11.519709 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:11.519692 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:42:11.519791 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:11.519731 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:42:11.572276 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:11.572257 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:21.043765 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:21.043724 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:21.144023 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:21.143989 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:25.250624 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:25.250585 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:25.353583 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:25.353548 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:31.525327 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:31.525300 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:42:31.529311 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:42:31.529282 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-68bb6996cc-xbbz9" Jul 10 18:43:02.852963 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:43:02.852880 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:02.953907 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:43:02.953878 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:14.717472 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:43:14.717441 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:14.781586 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:43:14.781554 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:18.178741 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:43:18.178705 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:18.255983 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:43:18.255951 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:25.260315 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:43:25.260275 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:25.359395 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:43:25.359371 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:29.472972 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:43:29.472933 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:34.916391 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:43:34.916358 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:43:42.531484 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:43:42.531441 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:14.435356 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:14.435319 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:14.488592 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:14.488570 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:50.102566 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:50.102534 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:53.711013 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:53.710979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:53.809187 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:53.809159 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:53.872379 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:53.872356 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:54.661097 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.661065 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7"] Jul 10 18:44:54.661401 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.661387 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baf2fb14-f51e-43cc-a8af-a5fc371113c8" containerName="registry" Jul 10 18:44:54.661450 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.661403 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="baf2fb14-f51e-43cc-a8af-a5fc371113c8" containerName="registry" Jul 10 18:44:54.661485 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.661461 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="baf2fb14-f51e-43cc-a8af-a5fc371113c8" containerName="registry" Jul 10 18:44:54.663565 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.663547 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:44:54.666856 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.666835 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 18:44:54.667640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.667622 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 18:44:54.667640 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.667626 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-k2nfw\"" Jul 10 18:44:54.674428 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.674405 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7"] Jul 10 18:44:54.766897 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.766874 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:44:54.767182 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.766908 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:44:54.767182 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.766931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbn84\" (UniqueName: \"kubernetes.io/projected/09092c99-2012-44e0-8b69-3fe6144619c4-kube-api-access-tbn84\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:44:54.867325 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.867304 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:44:54.867445 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.867336 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:44:54.867445 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.867356 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbn84\" (UniqueName: \"kubernetes.io/projected/09092c99-2012-44e0-8b69-3fe6144619c4-kube-api-access-tbn84\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:44:54.867719 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.867701 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:44:54.867796 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.867778 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:44:54.876892 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.876861 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbn84\" (UniqueName: \"kubernetes.io/projected/09092c99-2012-44e0-8b69-3fe6144619c4-kube-api-access-tbn84\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:44:54.974254 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:54.974194 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:44:55.097326 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:55.097304 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7"] Jul 10 18:44:55.099569 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:44:55.099545 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09092c99_2012_44e0_8b69_3fe6144619c4.slice/crio-b18614336faef915ec8f3827ff025fd724bfc856a4c51d039d762cfe638a478e WatchSource:0}: Error finding container b18614336faef915ec8f3827ff025fd724bfc856a4c51d039d762cfe638a478e: Status 404 returned error can't find the container with id b18614336faef915ec8f3827ff025fd724bfc856a4c51d039d762cfe638a478e Jul 10 18:44:55.533315 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:44:55.533284 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" event={"ID":"09092c99-2012-44e0-8b69-3fe6144619c4","Type":"ContainerStarted","Data":"b18614336faef915ec8f3827ff025fd724bfc856a4c51d039d762cfe638a478e"} Jul 10 18:45:00.558074 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:00.558016 2563 generic.go:356] "Generic (PLEG): container finished" podID="09092c99-2012-44e0-8b69-3fe6144619c4" containerID="0a76c85e78ac9ddfb865f3633f4c48593ad05755073df6edfd0223166f619927" exitCode=0 Jul 10 18:45:00.558387 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:00.558103 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" event={"ID":"09092c99-2012-44e0-8b69-3fe6144619c4","Type":"ContainerDied","Data":"0a76c85e78ac9ddfb865f3633f4c48593ad05755073df6edfd0223166f619927"} Jul 10 18:45:03.572683 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:03.572656 2563 generic.go:356] "Generic (PLEG): container finished" podID="09092c99-2012-44e0-8b69-3fe6144619c4" containerID="a4c68deb285fad8cd8fc6c5c2ec60b1340fb6cf7c7e36293342faee76a10d89e" exitCode=0 Jul 10 18:45:03.572991 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:03.572743 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" event={"ID":"09092c99-2012-44e0-8b69-3fe6144619c4","Type":"ContainerDied","Data":"a4c68deb285fad8cd8fc6c5c2ec60b1340fb6cf7c7e36293342faee76a10d89e"} Jul 10 18:45:11.815079 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:11.815046 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 18:45:11.816492 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:11.815520 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 18:45:11.821832 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:11.821813 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 18:45:12.610931 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:12.610898 2563 generic.go:356] "Generic (PLEG): container finished" podID="09092c99-2012-44e0-8b69-3fe6144619c4" containerID="91e3540865cd3aca97aeffe4d9b10d03f6b3e28450bf84b0b1f425f69027f8b0" exitCode=0 Jul 10 18:45:12.611080 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:12.610985 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" event={"ID":"09092c99-2012-44e0-8b69-3fe6144619c4","Type":"ContainerDied","Data":"91e3540865cd3aca97aeffe4d9b10d03f6b3e28450bf84b0b1f425f69027f8b0"} Jul 10 18:45:13.738322 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:13.738301 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:45:13.814234 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:13.814210 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-bundle\") pod \"09092c99-2012-44e0-8b69-3fe6144619c4\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " Jul 10 18:45:13.814341 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:13.814274 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-util\") pod \"09092c99-2012-44e0-8b69-3fe6144619c4\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " Jul 10 18:45:13.814341 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:13.814293 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbn84\" (UniqueName: \"kubernetes.io/projected/09092c99-2012-44e0-8b69-3fe6144619c4-kube-api-access-tbn84\") pod \"09092c99-2012-44e0-8b69-3fe6144619c4\" (UID: \"09092c99-2012-44e0-8b69-3fe6144619c4\") " Jul 10 18:45:13.814986 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:13.814960 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-bundle" (OuterVolumeSpecName: "bundle") pod "09092c99-2012-44e0-8b69-3fe6144619c4" (UID: "09092c99-2012-44e0-8b69-3fe6144619c4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:45:13.816336 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:13.816302 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09092c99-2012-44e0-8b69-3fe6144619c4-kube-api-access-tbn84" (OuterVolumeSpecName: "kube-api-access-tbn84") pod "09092c99-2012-44e0-8b69-3fe6144619c4" (UID: "09092c99-2012-44e0-8b69-3fe6144619c4"). InnerVolumeSpecName "kube-api-access-tbn84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:45:13.819382 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:13.819363 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-util" (OuterVolumeSpecName: "util") pod "09092c99-2012-44e0-8b69-3fe6144619c4" (UID: "09092c99-2012-44e0-8b69-3fe6144619c4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:45:13.915266 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:13.915248 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-bundle\") on node \"ip-10-0-137-86.ec2.internal\" DevicePath \"\"" Jul 10 18:45:13.915266 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:13.915266 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09092c99-2012-44e0-8b69-3fe6144619c4-util\") on node \"ip-10-0-137-86.ec2.internal\" DevicePath \"\"" Jul 10 18:45:13.915380 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:13.915276 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbn84\" (UniqueName: \"kubernetes.io/projected/09092c99-2012-44e0-8b69-3fe6144619c4-kube-api-access-tbn84\") on node \"ip-10-0-137-86.ec2.internal\" DevicePath \"\"" Jul 10 18:45:14.621043 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:14.620972 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" event={"ID":"09092c99-2012-44e0-8b69-3fe6144619c4","Type":"ContainerDied","Data":"b18614336faef915ec8f3827ff025fd724bfc856a4c51d039d762cfe638a478e"} Jul 10 18:45:14.621043 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:14.621009 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589z68v7" Jul 10 18:45:14.621043 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:14.621020 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b18614336faef915ec8f3827ff025fd724bfc856a4c51d039d762cfe638a478e" Jul 10 18:45:15.731159 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:15.731124 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:15.848263 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:15.848236 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:17.608663 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.608608 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx"] Jul 10 18:45:17.609047 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.608939 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09092c99-2012-44e0-8b69-3fe6144619c4" containerName="extract" Jul 10 18:45:17.609047 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.608952 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="09092c99-2012-44e0-8b69-3fe6144619c4" containerName="extract" Jul 10 18:45:17.609047 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.608971 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09092c99-2012-44e0-8b69-3fe6144619c4" containerName="pull" Jul 10 18:45:17.609047 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.608976 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="09092c99-2012-44e0-8b69-3fe6144619c4" containerName="pull" Jul 10 18:45:17.609047 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.608985 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09092c99-2012-44e0-8b69-3fe6144619c4" containerName="util" Jul 10 18:45:17.609047 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.608990 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="09092c99-2012-44e0-8b69-3fe6144619c4" containerName="util" Jul 10 18:45:17.609047 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.609048 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="09092c99-2012-44e0-8b69-3fe6144619c4" containerName="extract" Jul 10 18:45:17.651225 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.651198 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx"] Jul 10 18:45:17.651353 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.651287 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" Jul 10 18:45:17.653886 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.653866 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 10 18:45:17.653886 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.653882 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-cq6nd\"" Jul 10 18:45:17.654132 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.654117 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 10 18:45:17.654185 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.654137 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 10 18:45:17.741799 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.741772 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd9m9\" (UniqueName: \"kubernetes.io/projected/def43154-1289-46f8-878f-793a6defe375-kube-api-access-wd9m9\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx\" (UID: \"def43154-1289-46f8-878f-793a6defe375\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" Jul 10 18:45:17.741916 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.741823 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/def43154-1289-46f8-878f-793a6defe375-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx\" (UID: \"def43154-1289-46f8-878f-793a6defe375\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" Jul 10 18:45:17.842873 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.842830 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd9m9\" (UniqueName: \"kubernetes.io/projected/def43154-1289-46f8-878f-793a6defe375-kube-api-access-wd9m9\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx\" (UID: \"def43154-1289-46f8-878f-793a6defe375\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" Jul 10 18:45:17.842964 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.842896 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/def43154-1289-46f8-878f-793a6defe375-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx\" (UID: \"def43154-1289-46f8-878f-793a6defe375\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" Jul 10 18:45:17.845153 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.845134 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/def43154-1289-46f8-878f-793a6defe375-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx\" (UID: \"def43154-1289-46f8-878f-793a6defe375\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" Jul 10 18:45:17.851310 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.851288 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd9m9\" (UniqueName: \"kubernetes.io/projected/def43154-1289-46f8-878f-793a6defe375-kube-api-access-wd9m9\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx\" (UID: \"def43154-1289-46f8-878f-793a6defe375\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" Jul 10 18:45:17.961886 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:17.961866 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" Jul 10 18:45:18.120291 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:18.120263 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx"] Jul 10 18:45:18.123684 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:45:18.123659 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef43154_1289_46f8_878f_793a6defe375.slice/crio-c3515acf8ce00b3f94dbda9238a3a1ea6e3a8b35cf1fad92581402b13082a62d WatchSource:0}: Error finding container c3515acf8ce00b3f94dbda9238a3a1ea6e3a8b35cf1fad92581402b13082a62d: Status 404 returned error can't find the container with id c3515acf8ce00b3f94dbda9238a3a1ea6e3a8b35cf1fad92581402b13082a62d Jul 10 18:45:18.125661 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:18.125623 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:45:18.638554 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:18.638522 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" event={"ID":"def43154-1289-46f8-878f-793a6defe375","Type":"ContainerStarted","Data":"c3515acf8ce00b3f94dbda9238a3a1ea6e3a8b35cf1fad92581402b13082a62d"} Jul 10 18:45:21.426798 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.426766 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:21.653055 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.653025 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" event={"ID":"def43154-1289-46f8-878f-793a6defe375","Type":"ContainerStarted","Data":"0e6c45dbe9a9d5f40b070dda4fd2e51c72bc3018b9781019c2d287276eb7e434"} Jul 10 18:45:21.653210 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.653101 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" Jul 10 18:45:21.680290 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.680200 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" podStartSLOduration=1.873639152 podStartE2EDuration="4.68018629s" totalImagesPullingTime="2.806547138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:45:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:45:21.678412899 +0000 UTC m=+311.962964231" watchObservedRunningTime="2026-07-10 18:45:21.68018629 +0000 UTC m=+311.964737596" Jul 10 18:45:21.740037 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.740006 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:21.752150 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.752125 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-2mx9x"] Jul 10 18:45:21.754353 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.754338 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:21.756847 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.756833 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-btqtc\"" Jul 10 18:45:21.756847 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.756838 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 10 18:45:21.756969 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.756836 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 10 18:45:21.766780 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.766756 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-2mx9x"] Jul 10 18:45:21.770479 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.770462 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4f34b052-81cd-43af-b924-0873e8a11d20-certificates\") pod \"keda-operator-68bff59c-2mx9x\" (UID: \"4f34b052-81cd-43af-b924-0873e8a11d20\") " pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:21.770577 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.770502 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/4f34b052-81cd-43af-b924-0873e8a11d20-cabundle0\") pod \"keda-operator-68bff59c-2mx9x\" (UID: \"4f34b052-81cd-43af-b924-0873e8a11d20\") " pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:21.770577 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.770520 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv5d6\" (UniqueName: \"kubernetes.io/projected/4f34b052-81cd-43af-b924-0873e8a11d20-kube-api-access-tv5d6\") pod \"keda-operator-68bff59c-2mx9x\" (UID: \"4f34b052-81cd-43af-b924-0873e8a11d20\") " pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:21.871118 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.871095 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4f34b052-81cd-43af-b924-0873e8a11d20-certificates\") pod \"keda-operator-68bff59c-2mx9x\" (UID: \"4f34b052-81cd-43af-b924-0873e8a11d20\") " pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:21.871224 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.871142 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/4f34b052-81cd-43af-b924-0873e8a11d20-cabundle0\") pod \"keda-operator-68bff59c-2mx9x\" (UID: \"4f34b052-81cd-43af-b924-0873e8a11d20\") " pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:21.871224 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.871161 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv5d6\" (UniqueName: \"kubernetes.io/projected/4f34b052-81cd-43af-b924-0873e8a11d20-kube-api-access-tv5d6\") pod \"keda-operator-68bff59c-2mx9x\" (UID: \"4f34b052-81cd-43af-b924-0873e8a11d20\") " pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:21.871332 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:45:21.871235 2563 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found Jul 10 18:45:21.871332 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:45:21.871254 2563 secret.go:281] references non-existent secret key: ca.crt Jul 10 18:45:21.871332 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:45:21.871264 2563 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 10 18:45:21.871332 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:45:21.871276 2563 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-2mx9x: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 10 18:45:21.871332 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:45:21.871323 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f34b052-81cd-43af-b924-0873e8a11d20-certificates podName:4f34b052-81cd-43af-b924-0873e8a11d20 nodeName:}" failed. No retries permitted until 2026-07-10 18:45:22.371307979 +0000 UTC m=+312.655859265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/4f34b052-81cd-43af-b924-0873e8a11d20-certificates") pod "keda-operator-68bff59c-2mx9x" (UID: "4f34b052-81cd-43af-b924-0873e8a11d20") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 10 18:45:21.871752 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.871734 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/4f34b052-81cd-43af-b924-0873e8a11d20-cabundle0\") pod \"keda-operator-68bff59c-2mx9x\" (UID: \"4f34b052-81cd-43af-b924-0873e8a11d20\") " pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:21.884316 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.884296 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv5d6\" (UniqueName: \"kubernetes.io/projected/4f34b052-81cd-43af-b924-0873e8a11d20-kube-api-access-tv5d6\") pod \"keda-operator-68bff59c-2mx9x\" (UID: \"4f34b052-81cd-43af-b924-0873e8a11d20\") " pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:21.924693 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:21.924670 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:22.101561 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.101512 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:22.199606 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.199580 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-grtgb"] Jul 10 18:45:22.201828 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.201815 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:22.204156 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.204139 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 10 18:45:22.212838 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.212821 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-grtgb"] Jul 10 18:45:22.222547 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.222525 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:22.273225 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.273201 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/01846793-ce92-45ec-a948-586227fff537-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-grtgb\" (UID: \"01846793-ce92-45ec-a948-586227fff537\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:22.273335 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.273316 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb5r9\" (UniqueName: \"kubernetes.io/projected/01846793-ce92-45ec-a948-586227fff537-kube-api-access-qb5r9\") pod \"keda-metrics-apiserver-559f75f947-grtgb\" (UID: \"01846793-ce92-45ec-a948-586227fff537\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:22.273393 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.273355 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/01846793-ce92-45ec-a948-586227fff537-certificates\") pod \"keda-metrics-apiserver-559f75f947-grtgb\" (UID: \"01846793-ce92-45ec-a948-586227fff537\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:22.326799 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.326774 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:22.374097 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.374028 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb5r9\" (UniqueName: \"kubernetes.io/projected/01846793-ce92-45ec-a948-586227fff537-kube-api-access-qb5r9\") pod \"keda-metrics-apiserver-559f75f947-grtgb\" (UID: \"01846793-ce92-45ec-a948-586227fff537\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:22.374097 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.374069 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/01846793-ce92-45ec-a948-586227fff537-certificates\") pod \"keda-metrics-apiserver-559f75f947-grtgb\" (UID: \"01846793-ce92-45ec-a948-586227fff537\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:22.374256 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.374111 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/01846793-ce92-45ec-a948-586227fff537-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-grtgb\" (UID: \"01846793-ce92-45ec-a948-586227fff537\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:22.374256 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.374155 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4f34b052-81cd-43af-b924-0873e8a11d20-certificates\") pod \"keda-operator-68bff59c-2mx9x\" (UID: \"4f34b052-81cd-43af-b924-0873e8a11d20\") " pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:22.374459 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.374440 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/01846793-ce92-45ec-a948-586227fff537-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-grtgb\" (UID: \"01846793-ce92-45ec-a948-586227fff537\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:22.376416 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.376398 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4f34b052-81cd-43af-b924-0873e8a11d20-certificates\") pod \"keda-operator-68bff59c-2mx9x\" (UID: \"4f34b052-81cd-43af-b924-0873e8a11d20\") " pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:22.376728 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.376710 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/01846793-ce92-45ec-a948-586227fff537-certificates\") pod \"keda-metrics-apiserver-559f75f947-grtgb\" (UID: \"01846793-ce92-45ec-a948-586227fff537\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:22.381630 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.381610 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-xqzrd"] Jul 10 18:45:22.384794 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.384773 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" Jul 10 18:45:22.386137 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.386119 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb5r9\" (UniqueName: \"kubernetes.io/projected/01846793-ce92-45ec-a948-586227fff537-kube-api-access-qb5r9\") pod \"keda-metrics-apiserver-559f75f947-grtgb\" (UID: \"01846793-ce92-45ec-a948-586227fff537\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:22.387022 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.387008 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 10 18:45:22.395028 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.394997 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-xqzrd"] Jul 10 18:45:22.426103 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.426079 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:22.475546 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.475521 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc6j2\" (UniqueName: \"kubernetes.io/projected/e5267a3b-570d-4f02-a223-b71727c1b5b0-kube-api-access-mc6j2\") pod \"keda-admission-5fbd5c4644-xqzrd\" (UID: \"e5267a3b-570d-4f02-a223-b71727c1b5b0\") " pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" Jul 10 18:45:22.475888 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.475558 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e5267a3b-570d-4f02-a223-b71727c1b5b0-certificates\") pod \"keda-admission-5fbd5c4644-xqzrd\" (UID: \"e5267a3b-570d-4f02-a223-b71727c1b5b0\") " pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" Jul 10 18:45:22.485042 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.485014 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:22.511629 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.511594 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:22.576158 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.576128 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc6j2\" (UniqueName: \"kubernetes.io/projected/e5267a3b-570d-4f02-a223-b71727c1b5b0-kube-api-access-mc6j2\") pod \"keda-admission-5fbd5c4644-xqzrd\" (UID: \"e5267a3b-570d-4f02-a223-b71727c1b5b0\") " pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" Jul 10 18:45:22.576289 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.576196 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e5267a3b-570d-4f02-a223-b71727c1b5b0-certificates\") pod \"keda-admission-5fbd5c4644-xqzrd\" (UID: \"e5267a3b-570d-4f02-a223-b71727c1b5b0\") " pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" Jul 10 18:45:22.579250 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.579209 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e5267a3b-570d-4f02-a223-b71727c1b5b0-certificates\") pod \"keda-admission-5fbd5c4644-xqzrd\" (UID: \"e5267a3b-570d-4f02-a223-b71727c1b5b0\") " pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" Jul 10 18:45:22.584612 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.584566 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc6j2\" (UniqueName: \"kubernetes.io/projected/e5267a3b-570d-4f02-a223-b71727c1b5b0-kube-api-access-mc6j2\") pod \"keda-admission-5fbd5c4644-xqzrd\" (UID: \"e5267a3b-570d-4f02-a223-b71727c1b5b0\") " pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" Jul 10 18:45:22.661095 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.659992 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-grtgb"] Jul 10 18:45:22.662825 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:45:22.662802 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01846793_ce92_45ec_a948_586227fff537.slice/crio-3ecbf18c22d85bdd6126d4de4124d324689558ddb285fa22e5fb271f7b672499 WatchSource:0}: Error finding container 3ecbf18c22d85bdd6126d4de4124d324689558ddb285fa22e5fb271f7b672499: Status 404 returned error can't find the container with id 3ecbf18c22d85bdd6126d4de4124d324689558ddb285fa22e5fb271f7b672499 Jul 10 18:45:22.663821 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.663805 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:22.706900 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.706667 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" Jul 10 18:45:22.821060 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.821028 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-2mx9x"] Jul 10 18:45:22.824282 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:45:22.824231 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f34b052_81cd_43af_b924_0873e8a11d20.slice/crio-2a456e907c97bcab2bf9a628e8dc3b62dc0ce8218df6780691f263bdb212b294 WatchSource:0}: Error finding container 2a456e907c97bcab2bf9a628e8dc3b62dc0ce8218df6780691f263bdb212b294: Status 404 returned error can't find the container with id 2a456e907c97bcab2bf9a628e8dc3b62dc0ce8218df6780691f263bdb212b294 Jul 10 18:45:22.876064 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.875923 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-xqzrd"] Jul 10 18:45:22.878673 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:45:22.878624 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5267a3b_570d_4f02_a223_b71727c1b5b0.slice/crio-54dd87f3e17e2ae4c264929d52aa0605f3942e50d71ef1dd0f6da95ec4550681 WatchSource:0}: Error finding container 54dd87f3e17e2ae4c264929d52aa0605f3942e50d71ef1dd0f6da95ec4550681: Status 404 returned error can't find the container with id 54dd87f3e17e2ae4c264929d52aa0605f3942e50d71ef1dd0f6da95ec4550681 Jul 10 18:45:22.946529 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:22.946455 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:23.146076 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:23.146040 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:23.260778 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:23.260723 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:23.354101 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:23.354076 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:23.666081 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:23.666050 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" event={"ID":"e5267a3b-570d-4f02-a223-b71727c1b5b0","Type":"ContainerStarted","Data":"54dd87f3e17e2ae4c264929d52aa0605f3942e50d71ef1dd0f6da95ec4550681"} Jul 10 18:45:23.667369 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:23.667337 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-2mx9x" event={"ID":"4f34b052-81cd-43af-b924-0873e8a11d20","Type":"ContainerStarted","Data":"2a456e907c97bcab2bf9a628e8dc3b62dc0ce8218df6780691f263bdb212b294"} Jul 10 18:45:23.668582 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:23.668556 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" event={"ID":"01846793-ce92-45ec-a948-586227fff537","Type":"ContainerStarted","Data":"3ecbf18c22d85bdd6126d4de4124d324689558ddb285fa22e5fb271f7b672499"} Jul 10 18:45:25.680842 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:25.680807 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" event={"ID":"e5267a3b-570d-4f02-a223-b71727c1b5b0","Type":"ContainerStarted","Data":"f2df4ddca3cbb665a31b3b5174778e3568700ecb65c4d9d7202ce6dc28cb3902"} Jul 10 18:45:25.681248 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:25.680900 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" Jul 10 18:45:25.701113 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:25.701064 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" podStartSLOduration=1.770595419 podStartE2EDuration="3.701049943s" totalImagesPullingTime="1.930454524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:45:25.699605169 +0000 UTC m=+315.984156559" watchObservedRunningTime="2026-07-10 18:45:25.701049943 +0000 UTC m=+315.985601249" Jul 10 18:45:27.693217 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:27.693177 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-2mx9x" event={"ID":"4f34b052-81cd-43af-b924-0873e8a11d20","Type":"ContainerStarted","Data":"cdb92b7626f7b6e18c6c090e03e3c20957dbb0f93434d5ba0c9a14ba7a0d9bca"} Jul 10 18:45:27.693701 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:27.693292 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:45:27.694722 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:27.694700 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" event={"ID":"01846793-ce92-45ec-a948-586227fff537","Type":"ContainerStarted","Data":"602fb2b49c65c6eb5f339661b395c226e0ad8ff7add80784e33aedb380d0b33c"} Jul 10 18:45:27.694813 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:27.694760 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:27.710117 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:27.710075 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-2mx9x" podStartSLOduration=2.6093049390000003 podStartE2EDuration="6.710061369s" totalImagesPullingTime="4.10075643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:45:27.708789029 +0000 UTC m=+317.993340340" watchObservedRunningTime="2026-07-10 18:45:27.710061369 +0000 UTC m=+317.994612677" Jul 10 18:45:27.725940 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:27.725901 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" podStartSLOduration=1.46716714 podStartE2EDuration="5.725887732s" totalImagesPullingTime="4.258720592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:45:27.724584342 +0000 UTC m=+318.009135680" watchObservedRunningTime="2026-07-10 18:45:27.725887732 +0000 UTC m=+318.010439039" Jul 10 18:45:31.856603 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:31.856570 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:31.958387 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:31.958358 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:33.794329 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:33.794293 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:38.703848 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:38.703815 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-grtgb" Jul 10 18:45:42.663530 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:42.663500 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-lzsnx" Jul 10 18:45:46.689695 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:46.689664 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-xqzrd" Jul 10 18:45:48.701344 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:45:48.701303 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-2mx9x" Jul 10 18:46:14.495186 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:14.495098 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:15.629597 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:15.629558 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:30.519904 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.519862 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:30.629437 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.629408 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-rrnhp"] Jul 10 18:46:30.631684 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.631670 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:46:30.634298 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.634279 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-s8hkh\"" Jul 10 18:46:30.634712 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.634693 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 10 18:46:30.635156 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.635142 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 10 18:46:30.635156 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.635151 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 10 18:46:30.641563 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.641537 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:30.642302 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.642285 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-rrnhp"] Jul 10 18:46:30.719877 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.719853 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:30.758406 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.758380 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc727\" (UniqueName: \"kubernetes.io/projected/957e6b13-0f81-4a4d-a3a6-0e4789beca7e-kube-api-access-qc727\") pod \"llmisvc-controller-manager-67d95fdc48-rrnhp\" (UID: \"957e6b13-0f81-4a4d-a3a6-0e4789beca7e\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:46:30.758498 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.758424 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/957e6b13-0f81-4a4d-a3a6-0e4789beca7e-cert\") pod \"llmisvc-controller-manager-67d95fdc48-rrnhp\" (UID: \"957e6b13-0f81-4a4d-a3a6-0e4789beca7e\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:46:30.859272 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.859218 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/957e6b13-0f81-4a4d-a3a6-0e4789beca7e-cert\") pod \"llmisvc-controller-manager-67d95fdc48-rrnhp\" (UID: \"957e6b13-0f81-4a4d-a3a6-0e4789beca7e\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:46:30.859360 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.859274 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc727\" (UniqueName: \"kubernetes.io/projected/957e6b13-0f81-4a4d-a3a6-0e4789beca7e-kube-api-access-qc727\") pod \"llmisvc-controller-manager-67d95fdc48-rrnhp\" (UID: \"957e6b13-0f81-4a4d-a3a6-0e4789beca7e\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:46:30.859360 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:46:30.859352 2563 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 10 18:46:30.859441 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:46:30.859412 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/957e6b13-0f81-4a4d-a3a6-0e4789beca7e-cert podName:957e6b13-0f81-4a4d-a3a6-0e4789beca7e nodeName:}" failed. No retries permitted until 2026-07-10 18:46:31.359398033 +0000 UTC m=+381.643949323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/957e6b13-0f81-4a4d-a3a6-0e4789beca7e-cert") pod "llmisvc-controller-manager-67d95fdc48-rrnhp" (UID: "957e6b13-0f81-4a4d-a3a6-0e4789beca7e") : secret "llmisvc-webhook-server-cert" not found Jul 10 18:46:30.875536 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.875514 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc727\" (UniqueName: \"kubernetes.io/projected/957e6b13-0f81-4a4d-a3a6-0e4789beca7e-kube-api-access-qc727\") pod \"llmisvc-controller-manager-67d95fdc48-rrnhp\" (UID: \"957e6b13-0f81-4a4d-a3a6-0e4789beca7e\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:46:30.880210 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:30.880193 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:31.014663 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:31.014620 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:31.107885 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:31.107853 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:31.173659 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:31.173618 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:31.363130 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:31.363094 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/957e6b13-0f81-4a4d-a3a6-0e4789beca7e-cert\") pod \"llmisvc-controller-manager-67d95fdc48-rrnhp\" (UID: \"957e6b13-0f81-4a4d-a3a6-0e4789beca7e\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:46:31.365436 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:31.365414 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/957e6b13-0f81-4a4d-a3a6-0e4789beca7e-cert\") pod \"llmisvc-controller-manager-67d95fdc48-rrnhp\" (UID: \"957e6b13-0f81-4a4d-a3a6-0e4789beca7e\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:46:31.541577 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:31.541505 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:46:31.674736 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:31.674714 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-rrnhp"] Jul 10 18:46:31.677017 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:46:31.676990 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod957e6b13_0f81_4a4d_a3a6_0e4789beca7e.slice/crio-d8ce1da9636f5a2888a5177a6637a7cf9a43c54819d9f97ffb4d3c06a9d1b536 WatchSource:0}: Error finding container d8ce1da9636f5a2888a5177a6637a7cf9a43c54819d9f97ffb4d3c06a9d1b536: Status 404 returned error can't find the container with id d8ce1da9636f5a2888a5177a6637a7cf9a43c54819d9f97ffb4d3c06a9d1b536 Jul 10 18:46:31.968272 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:31.968240 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"d8ce1da9636f5a2888a5177a6637a7cf9a43c54819d9f97ffb4d3c06a9d1b536"} Jul 10 18:46:33.980092 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:33.980060 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"4ee7ab4d16104b60358b7069347e77de19a564b7acdf075c2a211ded8648992c"} Jul 10 18:46:33.980453 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:33.980181 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:46:34.016147 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:46:34.016071 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podStartSLOduration=1.949876406 podStartE2EDuration="4.016056351s" totalImagesPullingTime="2.066179945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:46:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:46:34.013762112 +0000 UTC m=+384.298313418" watchObservedRunningTime="2026-07-10 18:46:34.016056351 +0000 UTC m=+384.300607658" Jul 10 18:47:04.986580 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:04.986550 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:47:06.079985 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:06.079953 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:06.142872 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:06.142845 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:17.461584 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:17.461548 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:17.562467 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:17.562423 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:17.618288 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:17.618253 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:39.800875 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:39.800772 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:39.845546 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:39.845519 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-kdn5r"] Jul 10 18:47:39.848717 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:39.848702 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-kdn5r" Jul 10 18:47:39.851040 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:39.851023 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" Jul 10 18:47:39.851847 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:39.851833 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-r47hs\"" Jul 10 18:47:39.861597 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:39.861578 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-kdn5r"] Jul 10 18:47:39.884964 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:39.884944 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:39.959119 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:39.959096 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:39.998591 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:39.998570 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs6zn\" (UniqueName: \"kubernetes.io/projected/7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd-kube-api-access-bs6zn\") pod \"model-serving-api-fd65d7d6-kdn5r\" (UID: \"7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd\") " pod="kserve/model-serving-api-fd65d7d6-kdn5r" Jul 10 18:47:39.998697 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:39.998610 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd-tls-certs\") pod \"model-serving-api-fd65d7d6-kdn5r\" (UID: \"7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd\") " pod="kserve/model-serving-api-fd65d7d6-kdn5r" Jul 10 18:47:40.099268 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:40.099199 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs6zn\" (UniqueName: \"kubernetes.io/projected/7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd-kube-api-access-bs6zn\") pod \"model-serving-api-fd65d7d6-kdn5r\" (UID: \"7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd\") " pod="kserve/model-serving-api-fd65d7d6-kdn5r" Jul 10 18:47:40.099268 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:40.099246 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd-tls-certs\") pod \"model-serving-api-fd65d7d6-kdn5r\" (UID: \"7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd\") " pod="kserve/model-serving-api-fd65d7d6-kdn5r" Jul 10 18:47:40.101689 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:40.101667 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd-tls-certs\") pod \"model-serving-api-fd65d7d6-kdn5r\" (UID: \"7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd\") " pod="kserve/model-serving-api-fd65d7d6-kdn5r" Jul 10 18:47:40.110287 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:40.110266 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs6zn\" (UniqueName: \"kubernetes.io/projected/7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd-kube-api-access-bs6zn\") pod \"model-serving-api-fd65d7d6-kdn5r\" (UID: \"7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd\") " pod="kserve/model-serving-api-fd65d7d6-kdn5r" Jul 10 18:47:40.158763 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:40.158731 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-kdn5r" Jul 10 18:47:40.284206 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:40.284108 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-kdn5r"] Jul 10 18:47:40.286378 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:47:40.286351 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c00f2b9_0053_4eeb_b5ba_2d9f022a5cdd.slice/crio-8abb1f55ad0488ab3eea22994c69222c54dacad27f84f77a1da3f3233112b3cc WatchSource:0}: Error finding container 8abb1f55ad0488ab3eea22994c69222c54dacad27f84f77a1da3f3233112b3cc: Status 404 returned error can't find the container with id 8abb1f55ad0488ab3eea22994c69222c54dacad27f84f77a1da3f3233112b3cc Jul 10 18:47:40.432580 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:40.432557 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:40.534692 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:40.534641 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:40.630972 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:40.630948 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:41.281694 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:41.281637 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-kdn5r" event={"ID":"7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd","Type":"ContainerStarted","Data":"8abb1f55ad0488ab3eea22994c69222c54dacad27f84f77a1da3f3233112b3cc"} Jul 10 18:47:43.291557 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:43.291519 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-kdn5r" event={"ID":"7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd","Type":"ContainerStarted","Data":"84099e57f5f483bf4ec7ada839ba6e89c5f71262f7779d5bcf2242318380171a"} Jul 10 18:47:43.291939 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:43.291675 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-kdn5r" Jul 10 18:47:43.310145 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:43.310096 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-kdn5r" podStartSLOduration=1.964785665 podStartE2EDuration="4.310081314s" totalImagesPullingTime="2.345295649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:47:43.307952565 +0000 UTC m=+453.592503899" watchObservedRunningTime="2026-07-10 18:47:43.310081314 +0000 UTC m=+453.594632678" Jul 10 18:47:54.300947 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:54.300916 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-kdn5r" Jul 10 18:47:57.726441 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:57.726407 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:57.822293 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:47:57.822241 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:06.862172 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:06.862137 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:12.142101 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.142071 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-8bmx9"] Jul 10 18:48:12.145130 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.145115 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" Jul 10 18:48:12.148522 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.148497 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\"" Jul 10 18:48:12.148522 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.148509 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom\"" Jul 10 18:48:12.148718 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.148515 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-w4q5z\"" Jul 10 18:48:12.153088 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.153066 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-8bmx9"] Jul 10 18:48:12.240582 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.240552 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpdhm\" (UniqueName: \"kubernetes.io/projected/2f3e8d17-7c37-4c96-b733-1f2ccf5badef-kube-api-access-fpdhm\") pod \"seaweedfs-tls-custom-8b44d767d-8bmx9\" (UID: \"2f3e8d17-7c37-4c96-b733-1f2ccf5badef\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" Jul 10 18:48:12.240740 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.240618 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2f3e8d17-7c37-4c96-b733-1f2ccf5badef-data\") pod \"seaweedfs-tls-custom-8b44d767d-8bmx9\" (UID: \"2f3e8d17-7c37-4c96-b733-1f2ccf5badef\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" Jul 10 18:48:12.240740 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.240695 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/2f3e8d17-7c37-4c96-b733-1f2ccf5badef-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-8bmx9\" (UID: \"2f3e8d17-7c37-4c96-b733-1f2ccf5badef\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" Jul 10 18:48:12.341456 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.341428 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2f3e8d17-7c37-4c96-b733-1f2ccf5badef-data\") pod \"seaweedfs-tls-custom-8b44d767d-8bmx9\" (UID: \"2f3e8d17-7c37-4c96-b733-1f2ccf5badef\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" Jul 10 18:48:12.341563 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.341465 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/2f3e8d17-7c37-4c96-b733-1f2ccf5badef-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-8bmx9\" (UID: \"2f3e8d17-7c37-4c96-b733-1f2ccf5badef\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" Jul 10 18:48:12.341621 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.341605 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpdhm\" (UniqueName: \"kubernetes.io/projected/2f3e8d17-7c37-4c96-b733-1f2ccf5badef-kube-api-access-fpdhm\") pod \"seaweedfs-tls-custom-8b44d767d-8bmx9\" (UID: \"2f3e8d17-7c37-4c96-b733-1f2ccf5badef\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" Jul 10 18:48:12.341818 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.341799 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2f3e8d17-7c37-4c96-b733-1f2ccf5badef-data\") pod \"seaweedfs-tls-custom-8b44d767d-8bmx9\" (UID: \"2f3e8d17-7c37-4c96-b733-1f2ccf5badef\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" Jul 10 18:48:12.343781 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.343761 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/2f3e8d17-7c37-4c96-b733-1f2ccf5badef-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-8bmx9\" (UID: \"2f3e8d17-7c37-4c96-b733-1f2ccf5badef\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" Jul 10 18:48:12.349096 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.349077 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpdhm\" (UniqueName: \"kubernetes.io/projected/2f3e8d17-7c37-4c96-b733-1f2ccf5badef-kube-api-access-fpdhm\") pod \"seaweedfs-tls-custom-8b44d767d-8bmx9\" (UID: \"2f3e8d17-7c37-4c96-b733-1f2ccf5badef\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" Jul 10 18:48:12.455009 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.454940 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" Jul 10 18:48:12.582722 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:12.582625 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-8bmx9"] Jul 10 18:48:12.585025 ip-10-0-137-86 kubenswrapper[2563]: W0710 18:48:12.584998 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f3e8d17_7c37_4c96_b733_1f2ccf5badef.slice/crio-3b09201f1c298df86e8dc67db56df89c3229db7801c2bcfb91395e167881909f WatchSource:0}: Error finding container 3b09201f1c298df86e8dc67db56df89c3229db7801c2bcfb91395e167881909f: Status 404 returned error can't find the container with id 3b09201f1c298df86e8dc67db56df89c3229db7801c2bcfb91395e167881909f Jul 10 18:48:13.418660 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:13.418598 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" event={"ID":"2f3e8d17-7c37-4c96-b733-1f2ccf5badef","Type":"ContainerStarted","Data":"3b09201f1c298df86e8dc67db56df89c3229db7801c2bcfb91395e167881909f"} Jul 10 18:48:16.434079 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:16.434050 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" event={"ID":"2f3e8d17-7c37-4c96-b733-1f2ccf5badef","Type":"ContainerStarted","Data":"9d506eebd841f34e3d4785660318eb3df823a401dd129010afcf004bd7d17b4b"} Jul 10 18:48:16.454164 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:16.454121 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-8b44d767d-8bmx9" podStartSLOduration=1.027232042 podStartE2EDuration="4.454107471s" totalImagesPullingTime="3.426875429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:48:16.452269787 +0000 UTC m=+486.736821104" watchObservedRunningTime="2026-07-10 18:48:16.454107471 +0000 UTC m=+486.738658778" Jul 10 18:48:29.228541 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:29.228508 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:29.395562 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:29.395533 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:34.984933 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:34.984892 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.25:8081/readyz\": dial tcp 10.134.0.25:8081: connect: connection refused" Jul 10 18:48:35.516340 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:35.516304 2563 generic.go:356] "Generic (PLEG): container finished" podID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerID="4ee7ab4d16104b60358b7069347e77de19a564b7acdf075c2a211ded8648992c" exitCode=1 Jul 10 18:48:35.516340 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:35.516341 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerDied","Data":"4ee7ab4d16104b60358b7069347e77de19a564b7acdf075c2a211ded8648992c"} Jul 10 18:48:35.516604 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:35.516592 2563 scope.go:117] "RemoveContainer" containerID="4ee7ab4d16104b60358b7069347e77de19a564b7acdf075c2a211ded8648992c" Jul 10 18:48:36.522933 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:36.522897 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"d890dfe2cc2124686e6911e62b3358147fc4da60f066d13659f8503665e3fa28"} Jul 10 18:48:36.523409 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:36.523106 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:48:38.084313 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:38.084260 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:38.184598 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:38.184549 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:46.646975 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:46.646939 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:46.797941 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:46.797903 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:53.622169 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:48:53.622134 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:07.534026 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:49:07.533942 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:49:25.260101 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:49:25.260069 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:51.452124 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:49:51.452085 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:51.552302 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:49:51.552273 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:53.060140 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:49:53.060100 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:11.847986 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:50:11.847884 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 18:50:11.850064 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:50:11.848468 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 18:50:31.713681 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:50:31.713592 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:31.812919 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:50:31.812889 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:31.872232 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:50:31.872203 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:54.177699 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:50:54.177668 2563 generic.go:356] "Generic (PLEG): container finished" podID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerID="d890dfe2cc2124686e6911e62b3358147fc4da60f066d13659f8503665e3fa28" exitCode=1 Jul 10 18:50:54.177699 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:50:54.177681 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerDied","Data":"d890dfe2cc2124686e6911e62b3358147fc4da60f066d13659f8503665e3fa28"} Jul 10 18:50:54.178162 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:50:54.177747 2563 scope.go:117] "RemoveContainer" containerID="4ee7ab4d16104b60358b7069347e77de19a564b7acdf075c2a211ded8648992c" Jul 10 18:50:54.178162 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:50:54.178001 2563 scope.go:117] "RemoveContainer" containerID="d890dfe2cc2124686e6911e62b3358147fc4da60f066d13659f8503665e3fa28" Jul 10 18:50:54.178240 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:50:54.178180 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:50:57.531540 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:50:57.531511 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:50:57.532029 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:50:57.531835 2563 scope.go:117] "RemoveContainer" containerID="d890dfe2cc2124686e6911e62b3358147fc4da60f066d13659f8503665e3fa28" Jul 10 18:50:57.532091 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:50:57.532018 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:51:01.542018 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:51:01.541990 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:51:01.542381 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:51:01.542315 2563 scope.go:117] "RemoveContainer" containerID="d890dfe2cc2124686e6911e62b3358147fc4da60f066d13659f8503665e3fa28" Jul 10 18:51:01.542473 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:51:01.542453 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:51:16.334301 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:51:16.334268 2563 scope.go:117] "RemoveContainer" containerID="d890dfe2cc2124686e6911e62b3358147fc4da60f066d13659f8503665e3fa28" Jul 10 18:51:16.334700 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:51:16.334561 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:51:17.276319 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:51:17.276285 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"b0f5544debd8e228db02eac3320681833a7d40d6b49275ce3022be1fe54ef60a"} Jul 10 18:51:17.276521 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:51:17.276505 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:51:48.282756 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:51:48.282723 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:51:56.914277 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:51:56.914245 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:51:58.433294 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:51:58.433253 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:06.951469 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:52:06.951385 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:07.408499 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:52:07.408455 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:32.194181 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:32.194100 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:32.267569 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:32.267532 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:32.341740 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:32.341709 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:36.903740 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:36.903708 2563 generic.go:356] "Generic (PLEG): container finished" podID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerID="b0f5544debd8e228db02eac3320681833a7d40d6b49275ce3022be1fe54ef60a" exitCode=1 Jul 10 18:53:36.904117 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:36.903781 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerDied","Data":"b0f5544debd8e228db02eac3320681833a7d40d6b49275ce3022be1fe54ef60a"} Jul 10 18:53:36.904117 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:36.903862 2563 scope.go:117] "RemoveContainer" containerID="d890dfe2cc2124686e6911e62b3358147fc4da60f066d13659f8503665e3fa28" Jul 10 18:53:36.904203 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:36.904162 2563 scope.go:117] "RemoveContainer" containerID="b0f5544debd8e228db02eac3320681833a7d40d6b49275ce3022be1fe54ef60a" Jul 10 18:53:36.904340 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:53:36.904322 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:53:38.280506 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:38.280477 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:53:38.281026 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:38.280866 2563 scope.go:117] "RemoveContainer" containerID="b0f5544debd8e228db02eac3320681833a7d40d6b49275ce3022be1fe54ef60a" Jul 10 18:53:38.281121 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:53:38.281093 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:53:41.542500 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:41.542463 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:53:41.542907 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:41.542802 2563 scope.go:117] "RemoveContainer" containerID="b0f5544debd8e228db02eac3320681833a7d40d6b49275ce3022be1fe54ef60a" Jul 10 18:53:41.542970 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:53:41.542950 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:53:56.334405 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:53:56.334367 2563 scope.go:117] "RemoveContainer" containerID="b0f5544debd8e228db02eac3320681833a7d40d6b49275ce3022be1fe54ef60a" Jul 10 18:53:56.334897 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:53:56.334559 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:54:07.335029 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:54:07.334995 2563 scope.go:117] "RemoveContainer" containerID="b0f5544debd8e228db02eac3320681833a7d40d6b49275ce3022be1fe54ef60a" Jul 10 18:54:08.054847 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:54:08.054769 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"df5e8e833d4d4a13d6dae27a9887353b49d737fa50cb557eafbb3aeb33d1ebce"} Jul 10 18:54:08.054980 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:54:08.054965 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:54:39.061476 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:54:39.061442 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:54:44.571473 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:54:44.571436 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:55:02.531090 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:55:02.531013 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:55:02.625888 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:55:02.625862 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:55:03.065218 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:55:03.065181 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:55:11.879218 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:55:11.879188 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 18:55:11.881515 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:55:11.880963 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 18:56:27.695375 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:27.695341 2563 generic.go:356] "Generic (PLEG): container finished" podID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerID="df5e8e833d4d4a13d6dae27a9887353b49d737fa50cb557eafbb3aeb33d1ebce" exitCode=1 Jul 10 18:56:27.695848 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:27.695410 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerDied","Data":"df5e8e833d4d4a13d6dae27a9887353b49d737fa50cb557eafbb3aeb33d1ebce"} Jul 10 18:56:27.695848 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:27.695465 2563 scope.go:117] "RemoveContainer" containerID="b0f5544debd8e228db02eac3320681833a7d40d6b49275ce3022be1fe54ef60a" Jul 10 18:56:27.695848 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:27.695825 2563 scope.go:117] "RemoveContainer" containerID="df5e8e833d4d4a13d6dae27a9887353b49d737fa50cb557eafbb3aeb33d1ebce" Jul 10 18:56:27.696049 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:56:27.696027 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:56:29.060335 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:29.060303 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:56:29.060764 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:29.060670 2563 scope.go:117] "RemoveContainer" containerID="df5e8e833d4d4a13d6dae27a9887353b49d737fa50cb557eafbb3aeb33d1ebce" Jul 10 18:56:29.060865 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:56:29.060842 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:56:31.541929 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:31.541858 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:56:31.542284 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:31.542177 2563 scope.go:117] "RemoveContainer" containerID="df5e8e833d4d4a13d6dae27a9887353b49d737fa50cb557eafbb3aeb33d1ebce" Jul 10 18:56:31.542369 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:56:31.542347 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:56:33.150489 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:33.150454 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:56:33.226052 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:33.226020 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:56:33.600474 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:33.600392 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:56:47.334751 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:47.334718 2563 scope.go:117] "RemoveContainer" containerID="df5e8e833d4d4a13d6dae27a9887353b49d737fa50cb557eafbb3aeb33d1ebce" Jul 10 18:56:47.335138 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:56:47.334886 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:56:51.597011 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:56:51.596967 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:57:01.334756 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:57:01.334718 2563 scope.go:117] "RemoveContainer" containerID="df5e8e833d4d4a13d6dae27a9887353b49d737fa50cb557eafbb3aeb33d1ebce" Jul 10 18:57:01.335284 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:57:01.334895 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:57:16.334686 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:57:16.334628 2563 scope.go:117] "RemoveContainer" containerID="df5e8e833d4d4a13d6dae27a9887353b49d737fa50cb557eafbb3aeb33d1ebce" Jul 10 18:57:16.335129 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:57:16.334949 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:57:16.932090 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:57:16.932060 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115"} Jul 10 18:57:16.932297 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:57:16.932277 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:57:47.939539 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:57:47.939505 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:59:21.132929 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:21.132895 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:34.570929 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:34.570894 2563 generic.go:356] "Generic (PLEG): container finished" podID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerID="254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115" exitCode=1 Jul 10 18:59:34.571340 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:34.570959 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerDied","Data":"254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115"} Jul 10 18:59:34.571340 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:34.571042 2563 scope.go:117] "RemoveContainer" containerID="df5e8e833d4d4a13d6dae27a9887353b49d737fa50cb557eafbb3aeb33d1ebce" Jul 10 18:59:34.571340 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:34.571308 2563 scope.go:117] "RemoveContainer" containerID="254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115" Jul 10 18:59:34.571498 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:59:34.571479 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:59:37.937109 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:37.937074 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:59:37.937570 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:37.937408 2563 scope.go:117] "RemoveContainer" containerID="254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115" Jul 10 18:59:37.937629 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:59:37.937598 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:59:41.541979 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:41.541949 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 18:59:41.542360 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:41.542265 2563 scope.go:117] "RemoveContainer" containerID="254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115" Jul 10 18:59:41.542440 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:59:41.542422 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 18:59:43.342485 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:43.342453 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:43.443558 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:43.443531 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:43.639018 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:43.638967 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:55.334619 ip-10-0-137-86 kubenswrapper[2563]: I0710 18:59:55.334591 2563 scope.go:117] "RemoveContainer" containerID="254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115" Jul 10 18:59:55.335126 ip-10-0-137-86 kubenswrapper[2563]: E0710 18:59:55.334788 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:00:08.334768 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:00:08.334731 2563 scope.go:117] "RemoveContainer" containerID="254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115" Jul 10 19:00:08.335254 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:00:08.334902 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:00:11.906081 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:00:11.905979 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:00:11.909886 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:00:11.909756 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:00:13.500696 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:00:13.500659 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:00:13.624862 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:00:13.624825 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:00:13.695662 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:00:13.695601 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:00:20.336934 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:00:20.336904 2563 scope.go:117] "RemoveContainer" containerID="254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115" Jul 10 19:00:20.337320 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:00:20.337070 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:00:34.334160 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:00:34.334129 2563 scope.go:117] "RemoveContainer" containerID="254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115" Jul 10 19:00:34.334766 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:00:34.334304 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:00:45.334973 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:00:45.334947 2563 scope.go:117] "RemoveContainer" containerID="254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115" Jul 10 19:00:45.335331 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:00:45.335091 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:00:57.334419 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:00:57.334390 2563 scope.go:117] "RemoveContainer" containerID="254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115" Jul 10 19:01:03.660565 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:01:03.660473 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:01:03.752293 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:01:03.752262 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:01:03.915777 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:01:03.915699 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:01:13.997184 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:01:13.997152 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c"} Jul 10 19:01:13.997568 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:01:13.997352 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:01:45.003291 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:01:45.003264 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:02:05.809872 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:02:05.809839 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:02:23.865520 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:02:23.865485 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:02:23.969219 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:02:23.969181 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:02:24.152160 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:02:24.152136 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:03:29.572367 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:29.572279 2563 generic.go:356] "Generic (PLEG): container finished" podID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" exitCode=1 Jul 10 19:03:29.572367 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:29.572354 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerDied","Data":"37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c"} Jul 10 19:03:29.572857 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:29.572417 2563 scope.go:117] "RemoveContainer" containerID="254436bbfcd986641aa3814fb150734cece87ddc980bc05539d594c50c552115" Jul 10 19:03:29.572857 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:29.572752 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:03:29.572944 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:03:29.572921 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:03:30.002014 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:30.001966 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:03:30.578896 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:30.578865 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:03:30.579362 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:03:30.579098 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:03:31.542542 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:31.542502 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:03:31.588822 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:31.588799 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:03:31.589165 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:03:31.588996 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:03:35.431906 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:35.431878 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:03:35.522385 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:35.522356 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:03:35.609434 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:35.609405 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:03:43.334828 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:43.334800 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:03:43.335207 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:03:43.334961 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:03:58.334325 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:03:58.334294 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:03:58.334830 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:03:58.334457 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:04:10.336890 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:04:10.336814 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:04:10.337326 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:04:10.337033 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:04:22.334500 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:04:22.334468 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:04:22.334996 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:04:22.334639 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:04:37.137471 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:04:37.137439 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:04:37.334826 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:04:37.334799 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:04:37.335010 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:04:37.334989 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:04:38.998203 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:04:38.998162 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:04:49.334533 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:04:49.334505 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:04:49.334919 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:04:49.334683 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:05:00.336408 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:05:00.336379 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:05:00.336875 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:05:00.336542 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:05:11.334747 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:05:11.334722 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:05:11.335174 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:05:11.334883 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:05:11.932767 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:05:11.932665 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:05:11.936760 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:05:11.936545 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:05:26.335014 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:05:26.334984 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:05:26.335440 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:05:26.335151 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:05:37.334847 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:05:37.334770 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:05:37.335199 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:05:37.334929 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:05:39.696283 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:05:39.696247 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:05:39.796855 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:05:39.796824 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:05:52.334777 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:05:52.334750 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:05:52.335192 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:05:52.334935 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:06:03.035438 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:06:03.035405 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:06:03.334365 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:06:03.334284 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:06:03.334513 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:06:03.334431 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:06:16.334172 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:06:16.334144 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:06:16.334638 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:06:16.334428 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:06:17.303946 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:06:17.303906 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e"} Jul 10 19:06:17.304130 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:06:17.304109 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:06:48.310301 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:06:48.310268 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:07:10.195000 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:07:10.194929 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:07:10.323374 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:07:10.323337 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:07:10.592837 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:07:10.592755 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:08:34.252685 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:34.252590 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:08:35.916162 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:35.916129 2563 generic.go:356] "Generic (PLEG): container finished" podID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" exitCode=1 Jul 10 19:08:35.916557 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:35.916184 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerDied","Data":"195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e"} Jul 10 19:08:35.916557 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:35.916238 2563 scope.go:117] "RemoveContainer" containerID="37571f533f7dc37ec0db8366635cf797a519411dd9e5a31638fff0427454c66c" Jul 10 19:08:35.916557 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:35.916528 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:08:35.916765 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:08:35.916742 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:08:38.309441 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:38.309408 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:08:38.309848 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:38.309722 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:08:38.309896 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:08:38.309879 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:08:41.317776 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:41.317736 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:08:41.404456 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:41.404429 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:08:41.484060 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:41.484029 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:08:41.542017 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:41.541996 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:08:41.542360 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:41.542347 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:08:41.542520 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:08:41.542503 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:08:55.334688 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:08:55.334639 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:08:55.335079 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:08:55.334821 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:09:07.334515 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:09:07.334481 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:09:07.335066 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:09:07.334697 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:09:21.334635 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:09:21.334603 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:09:21.335074 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:09:21.334808 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:09:34.339396 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:09:34.339369 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:09:34.339789 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:09:34.339515 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:09:47.334801 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:09:47.334773 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:09:47.335299 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:09:47.334937 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:10:02.334768 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:10:02.334693 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:10:02.335209 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:10:02.334856 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:10:11.960016 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:10:11.959915 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:10:11.965884 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:10:11.964521 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:10:12.654061 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:10:12.654033 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:10:13.334533 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:10:13.334507 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:10:13.334876 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:10:13.334700 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:10:13.890450 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:10:13.890418 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:10:28.334738 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:10:28.334705 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:10:28.335317 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:10:28.334865 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:10:43.335048 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:10:43.335019 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:10:43.335437 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:10:43.335173 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:10:57.334733 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:10:57.334700 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:10:57.335151 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:10:57.334852 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:11:12.335115 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:12.335086 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:11:12.335566 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:11:12.335256 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:11:27.334992 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:27.334960 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:11:27.335376 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:11:27.335090 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:11:32.778401 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:32.778323 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:11:32.880804 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:32.880766 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:11:41.334934 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:41.334899 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:11:41.335364 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:11:41.335069 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:11:48.923214 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:48.923171 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:11:49.025294 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:49.025261 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:11:49.162777 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:49.162749 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:11:56.334738 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:56.334706 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:11:56.335121 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:11:56.334929 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:11:59.017603 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:59.017567 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:11:59.125692 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:59.125666 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:11:59.230446 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:11:59.230420 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:12:10.337025 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:12:10.336991 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:12:10.337493 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:12:10.337210 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:12:21.334292 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:12:21.334259 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:12:21.334867 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:12:21.334467 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:12:36.334687 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:12:36.334638 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:12:36.335095 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:12:36.334829 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:12:49.334638 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:12:49.334610 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:12:49.335044 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:12:49.334788 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:13:00.336973 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:13:00.336945 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:13:00.337368 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:13:00.337120 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:13:13.334844 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:13:13.334754 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:13:13.335344 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:13:13.334918 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:13:24.334800 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:13:24.334769 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:13:24.335199 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:13:24.334949 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:13:35.334043 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:13:35.334016 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:13:35.334421 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:13:35.334296 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:13:36.242105 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:13:36.242068 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd"} Jul 10 19:13:36.242325 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:13:36.242275 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:13:50.323468 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:13:50.323438 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:13:50.436767 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:13:50.436729 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:13:50.547377 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:13:50.547337 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:14:07.248707 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:14:07.248665 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:14:34.770582 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:14:34.770499 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:14:34.825756 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:14:34.825723 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:15:11.987206 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:15:11.987094 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:15:11.993453 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:15:11.993434 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:15:20.422667 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:15:20.422618 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:15:21.880399 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:15:21.880363 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:15:55.868198 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:15:55.868131 2563 generic.go:356] "Generic (PLEG): container finished" podID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" exitCode=1 Jul 10 19:15:55.868635 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:15:55.868206 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerDied","Data":"3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd"} Jul 10 19:15:55.868635 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:15:55.868268 2563 scope.go:117] "RemoveContainer" containerID="195c38e89fb814a31ab55deaeab081396ef7ae89c531fcaf3d22d45cfe31342e" Jul 10 19:15:55.868635 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:15:55.868559 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:15:55.868853 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:15:55.868766 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:15:57.246933 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:15:57.246899 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:15:57.247406 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:15:57.247326 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:15:57.247547 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:15:57.247525 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:16:01.541822 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:16:01.541736 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:16:01.542217 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:16:01.542076 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:16:01.542266 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:16:01.542242 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:16:02.709819 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:16:02.709780 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:16:12.334628 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:16:12.334593 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:16:12.335099 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:16:12.334807 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:16:24.335008 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:16:24.334975 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:16:24.335376 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:16:24.335152 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:16:38.339127 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:16:38.339102 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:16:38.339545 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:16:38.339256 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:16:49.335031 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:16:49.334997 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:16:49.335549 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:16:49.335223 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:16:50.904257 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:16:50.904218 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:16:51.002149 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:16:51.002119 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:16:51.409961 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:16:51.409931 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:17:01.334835 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:17:01.334804 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:17:01.335295 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:17:01.335013 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:17:01.759131 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:17:01.759098 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:17:12.334091 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:17:12.334063 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:17:12.334475 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:17:12.334238 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:17:24.334800 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:17:24.334772 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:17:24.335159 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:17:24.334960 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:17:38.334553 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:17:38.334521 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:17:38.335136 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:17:38.334737 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:17:53.334853 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:17:53.334821 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:17:53.335261 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:17:53.334981 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:18:07.335093 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:18:07.335057 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:18:07.335596 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:18:07.335270 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:18:18.334219 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:18:18.334179 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:18:18.334718 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:18:18.334347 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:18:21.102528 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:18:21.102490 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:18:21.188699 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:18:21.188661 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:18:21.389451 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:18:21.389421 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:18:31.334469 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:18:31.334442 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:18:31.334902 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:18:31.334596 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:18:44.334187 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:18:44.334157 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:18:44.334681 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:18:44.334317 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:18:55.334088 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:18:55.334062 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:18:55.334548 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:18:55.334222 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:19:07.334834 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:19:07.334750 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:19:07.335274 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:19:07.334947 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:19:18.334950 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:19:18.334916 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:19:18.335406 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:19:18.335132 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:19:31.335022 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:19:31.334984 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:19:31.335505 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:19:31.335198 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:19:41.285498 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:19:41.285465 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:19:41.402405 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:19:41.402362 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:19:41.594071 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:19:41.594001 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:19:43.334584 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:19:43.334552 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:19:43.334981 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:19:43.334761 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:19:57.334960 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:19:57.334933 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:19:57.335331 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:19:57.335109 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:20:09.334552 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:20:09.334524 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:20:09.335015 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:20:09.334702 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:20:12.014824 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:20:12.014724 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:20:12.024316 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:20:12.024299 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:20:23.334525 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:20:23.334494 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:20:23.334930 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:20:23.334683 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:20:38.334798 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:20:38.334769 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:20:38.335160 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:20:38.334922 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:20:52.334850 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:20:52.334822 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:20:52.335315 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:20:52.334968 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:21:01.513479 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:21:01.513450 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:21:03.334430 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:21:03.334395 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:21:03.334892 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:21:03.334710 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:21:04.053673 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:21:04.053580 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:21:04.193229 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:21:04.193201 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925"} Jul 10 19:21:04.193409 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:21:04.193394 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:21:35.201240 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:21:35.201210 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:22:23.782945 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:22:23.782864 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:22:23.873520 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:22:23.873496 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:22:24.070880 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:22:24.070817 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:23:04.221012 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:04.220965 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:23:21.542436 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:21.542344 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.25:8081/healthz\": dial tcp 10.134.0.25:8081: connect: connection refused" Jul 10 19:23:21.782262 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:21.782230 2563 generic.go:356] "Generic (PLEG): container finished" podID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" exitCode=1 Jul 10 19:23:21.782449 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:21.782309 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerDied","Data":"aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925"} Jul 10 19:23:21.782449 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:21.782379 2563 scope.go:117] "RemoveContainer" containerID="3c5af397ae1639901d3e732a2cf96db467a16b2f3b22d73562d73d06aa4940cd" Jul 10 19:23:21.782748 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:21.782733 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:23:21.782981 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:23:21.782952 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:23:23.969510 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:23.969480 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:23:24.035863 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:24.035832 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:23:24.219078 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:24.219041 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:23:25.199208 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:25.199172 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:23:25.199635 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:25.199479 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:23:25.199724 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:23:25.199666 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:23:31.542198 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:31.542119 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:23:31.542551 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:31.542504 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:23:31.542725 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:23:31.542704 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:23:43.335018 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:43.334981 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:23:43.335530 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:23:43.335230 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:23:56.335678 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:23:56.335557 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:23:56.336154 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:23:56.335911 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:24:01.246510 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:24:01.246475 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:24:01.341982 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:24:01.341951 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:24:01.976910 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:24:01.976874 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:24:07.334333 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:24:07.334300 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:24:07.334840 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:24:07.334490 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:24:20.337205 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:24:20.337171 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:24:20.337666 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:24:20.337408 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:24:33.334946 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:24:33.334918 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:24:33.335316 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:24:33.335090 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:24:46.335028 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:24:46.335002 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:24:46.335495 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:24:46.335163 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:24:47.268265 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:24:47.268225 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:24:57.334276 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:24:57.334243 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:24:57.334686 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:24:57.334412 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:25:01.469456 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:25:01.469377 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:25:01.569961 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:25:01.569932 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:25:01.801189 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:25:01.801097 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:25:11.334174 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:25:11.334140 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:25:11.334641 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:25:11.334387 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:25:12.045473 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:25:12.045362 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:25:12.055054 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:25:12.055032 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:25:26.334561 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:25:26.334528 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:25:26.335178 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:25:26.334827 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:25:38.334685 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:25:38.334634 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:25:38.335073 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:25:38.334833 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:25:46.736988 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:25:46.736956 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:25:46.810265 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:25:46.810226 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:25:51.334239 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:25:51.334206 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:25:51.334607 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:25:51.334377 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:26:06.334725 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:26:06.334696 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:26:06.335202 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:26:06.334872 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:26:20.336487 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:26:20.336450 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:26:20.336997 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:26:20.336701 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:26:21.717693 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:26:21.717661 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:26:23.016422 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:26:23.016382 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:26:34.334780 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:26:34.334700 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:26:34.335179 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:26:34.334936 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:26:48.335252 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:26:48.335217 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:26:48.335879 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:26:48.335388 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:27:01.334832 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:27:01.334802 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:27:01.335273 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:27:01.335027 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:27:16.334741 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:27:16.334704 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:27:16.335127 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:27:16.334927 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:27:31.334928 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:27:31.334899 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:27:31.335407 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:27:31.335081 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:27:41.855529 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:27:41.855484 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:27:41.922459 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:27:41.922429 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:27:42.196363 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:27:42.196313 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:27:45.335043 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:27:45.335010 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:27:45.335417 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:27:45.335196 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:27:59.335054 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:27:59.335022 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:27:59.335445 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:27:59.335188 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:28:12.334528 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:28:12.334454 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:28:12.335020 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:28:12.334680 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:28:21.971912 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:28:21.971883 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:28:22.050841 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:28:22.050810 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:28:22.108262 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:28:22.108231 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:28:26.334761 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:28:26.334730 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:28:26.335239 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:28:26.335106 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:28:27.135881 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:28:27.135849 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645"} Jul 10 19:28:27.136074 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:28:27.136055 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:28:58.142838 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:28:58.142807 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:29:02.170514 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:29:02.170470 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:29:02.263111 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:29:02.263064 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:29:02.323535 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:29:02.323507 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:29:40.146289 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:29:40.146168 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:30:12.081112 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:30:12.080979 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:30:12.089395 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:30:12.089375 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:30:45.753731 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:30:45.753631 2563 generic.go:356] "Generic (PLEG): container finished" podID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" exitCode=1 Jul 10 19:30:45.753731 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:30:45.753710 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerDied","Data":"28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645"} Jul 10 19:30:45.754224 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:30:45.753800 2563 scope.go:117] "RemoveContainer" containerID="aafdeaccaf023e9d9000d36f9f11013347b90e63d390933af7869fca76e2a925" Jul 10 19:30:45.754224 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:30:45.754144 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:30:45.754389 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:30:45.754366 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:30:48.141059 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:30:48.141030 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:30:48.141447 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:30:48.141336 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:30:48.141532 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:30:48.141508 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:30:51.542336 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:30:51.542296 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:30:51.542756 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:30:51.542623 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:30:51.542821 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:30:51.542799 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:31:04.335161 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:31:04.335083 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:31:04.335519 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:31:04.335263 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:31:19.334536 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:31:19.334507 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:31:19.335038 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:31:19.334693 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:31:23.850364 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:31:23.850327 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:31:25.087454 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:31:25.087417 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:31:30.336545 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:31:30.336514 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:31:30.336949 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:31:30.336744 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:31:43.334242 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:31:43.334204 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:31:43.334671 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:31:43.334382 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:31:56.334320 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:31:56.334288 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:31:56.334873 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:31:56.334449 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:32:07.334902 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:32:07.334862 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:32:07.335364 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:32:07.335096 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:32:18.334769 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:32:18.334731 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:32:18.335272 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:32:18.334881 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:32:30.447110 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:32:30.447073 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:32:33.334113 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:32:33.334039 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:32:33.334454 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:32:33.334244 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:32:44.335218 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:32:44.335183 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:32:44.335626 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:32:44.335393 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:32:54.018368 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:32:54.018331 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:32:54.085449 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:32:54.085414 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:32:54.143902 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:32:54.143874 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:32:56.334633 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:32:56.334594 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:32:56.335196 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:32:56.334800 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:33:07.334526 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:33:07.334494 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:33:07.335050 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:33:07.334719 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:33:18.334375 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:33:18.334333 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:33:18.334897 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:33:18.334514 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:33:30.336446 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:33:30.336393 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:33:30.336986 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:33:30.336578 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:33:44.177994 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:33:44.177959 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:33:44.259447 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:33:44.259416 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:33:44.334310 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:33:44.334279 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:33:44.334475 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:33:44.334443 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:33:44.746393 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:33:44.746358 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:33:58.334760 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:33:58.334721 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:33:58.335250 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:33:58.334914 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:34:13.334953 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:34:13.334869 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:34:13.335413 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:34:13.335029 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:34:26.334894 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:34:26.334861 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:34:26.335285 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:34:26.335021 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:34:26.354183 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:34:26.354151 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:34:34.352899 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:34:34.352866 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:34:34.670070 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:34:34.670035 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:34:34.770795 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:34:34.770759 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:34:40.340294 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:34:40.340270 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:34:40.340686 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:34:40.340424 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:34:55.334911 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:34:55.334883 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:34:55.335365 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:34:55.335052 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:35:10.336396 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:35:10.336362 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:35:10.336885 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:35:10.336588 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:35:12.108156 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:35:12.108051 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:35:12.117061 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:35:12.117045 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:35:24.339811 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:35:24.339779 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:35:24.340313 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:35:24.340010 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:35:39.334767 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:35:39.334684 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:35:39.335285 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:35:39.334923 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:35:45.227613 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:35:45.227556 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:35:45.288854 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:35:45.288822 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:35:54.334403 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:35:54.334371 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:35:54.334840 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:35:54.334682 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:36:10.241300 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:36:10.241270 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5"} Jul 10 19:36:10.241795 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:36:10.241482 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:36:41.248312 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:36:41.248279 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:36:44.710747 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:36:44.710717 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:36:45.832902 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:36:45.832860 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:37:54.868588 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:37:54.868553 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:37:54.938062 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:37:54.938024 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:37:55.184051 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:37:55.184020 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:38:20.071406 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:38:20.071376 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:38:29.864003 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:38:29.863967 2563 generic.go:356] "Generic (PLEG): container finished" podID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" exitCode=1 Jul 10 19:38:29.864450 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:38:29.864020 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerDied","Data":"111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5"} Jul 10 19:38:29.864450 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:38:29.864068 2563 scope.go:117] "RemoveContainer" containerID="28d917ffdfc92766a8e203ffa3d4428c38bb83cb78efe174b3184ca9d7d79645" Jul 10 19:38:29.864450 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:38:29.864380 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:38:29.864606 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:38:29.864585 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:38:31.246329 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:38:31.246241 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:38:31.246740 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:38:31.246590 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:38:31.246824 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:38:31.246802 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:38:31.542537 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:38:31.542450 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:38:31.875256 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:38:31.875180 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:38:31.875400 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:38:31.875352 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:38:43.335011 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:38:43.334979 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:38:43.335418 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:38:43.335154 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:38:54.334433 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:38:54.334401 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:38:54.334827 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:38:54.334573 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:39:07.334230 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:39:07.334190 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:39:07.334817 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:39:07.334374 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:39:15.062082 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:39:15.062049 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:39:15.149842 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:39:15.149803 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:39:15.221500 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:39:15.221471 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:39:20.337244 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:39:20.337209 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:39:20.337681 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:39:20.337444 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:39:33.335101 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:39:33.335066 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:39:33.335579 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:39:33.335265 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:39:48.334368 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:39:48.334336 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:39:48.334935 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:39:48.334567 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:40:00.336903 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:00.336874 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:40:00.337301 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:40:00.337077 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:40:12.142419 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:12.142392 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:40:12.150765 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:12.150745 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:40:12.334639 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:12.334608 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:40:12.334872 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:40:12.334847 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:40:23.335097 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:23.335059 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:40:23.335684 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:40:23.335295 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:40:25.230921 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:25.230887 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:40:26.253309 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:26.253275 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:40:26.605617 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:26.605508 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:40:34.334430 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:34.334395 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:40:34.334899 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:40:34.334561 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:40:36.289273 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:36.289234 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:40:37.330920 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:37.330883 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:40:37.469288 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:37.469257 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:40:49.335140 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:40:49.335113 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:40:49.335500 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:40:49.335255 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:41:03.334543 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:41:03.334504 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:41:03.334986 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:41:03.334700 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:41:14.334518 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:41:14.334479 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:41:14.334965 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:41:14.334725 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:41:29.334763 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:41:29.334733 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:41:29.335268 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:41:29.334907 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:41:42.334772 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:41:42.334695 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:41:42.335127 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:41:42.334859 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:41:47.433999 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:41:47.433959 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:41:49.088802 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:41:49.088730 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:41:52.625455 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:41:52.625415 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:41:56.335219 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:41:56.335185 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:41:56.335799 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:41:56.335361 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:41:58.479042 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:41:58.478998 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:41:59.526722 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:41:59.526680 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:41:59.878412 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:41:59.878385 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:42:11.334609 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:42:11.334575 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:42:11.335151 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:42:11.334795 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:42:26.339754 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:42:26.339725 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:42:26.340131 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:42:26.339895 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:42:40.336295 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:42:40.336257 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:42:40.336753 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:42:40.336452 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:42:54.334805 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:42:54.334774 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:42:54.335295 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:42:54.334955 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:43:08.334996 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:08.334899 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:43:08.335428 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:43:08.335077 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:43:19.629859 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:19.629819 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:43:20.617719 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:20.617681 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:43:21.010676 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:21.010626 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:43:21.334276 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:21.334205 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:43:21.334418 ip-10-0-137-86 kubenswrapper[2563]: E0710 19:43:21.334379 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-67d95fdc48-rrnhp_kserve(957e6b13-0f81-4a4d-a3a6-0e4789beca7e)\"" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" podUID="957e6b13-0f81-4a4d-a3a6-0e4789beca7e" Jul 10 19:43:30.667310 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:30.667258 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:43:32.794808 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:32.794775 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:43:32.895751 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:32.895722 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:43:36.346321 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:36.346293 2563 scope.go:117] "RemoveContainer" containerID="111dd8aa53664a3ac3bf549f30219249ccb358aa9b2c532d636d8954372a4ae5" Jul 10 19:43:36.346736 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:36.346556 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 19:43:37.248219 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:37.248184 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" event={"ID":"957e6b13-0f81-4a4d-a3a6-0e4789beca7e","Type":"ContainerStarted","Data":"569a4cbf36cfb26d31213c35221e76cd7f9c59e34e219fe81aa88c6ab2632c6b"} Jul 10 19:43:37.248426 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:43:37.248408 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:44:04.127413 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:04.127376 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:44:04.282167 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:04.282134 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:44:04.342402 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:04.342377 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:44:07.233340 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:07.233308 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dj7wf_ec2f7adf-fe91-4da9-b03e-b33b6f2e4fd1/global-pull-secret-syncer/0.log" Jul 10 19:44:07.296494 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:07.296465 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" Jul 10 19:44:07.436413 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:07.436391 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-86.ec2.internal_f43f038d238bfb0aa554060566ad6e31/haproxy/0.log" Jul 10 19:44:08.255261 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:08.255234 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-rrnhp" Jul 10 19:44:09.761034 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:09.760997 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:44:09.860878 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:09.860848 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:44:11.665409 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:11.665381 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e879a6-6b80-4809-983b-fcca92cc36f7/alertmanager/0.log" Jul 10 19:44:11.694589 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:11.694559 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e879a6-6b80-4809-983b-fcca92cc36f7/config-reloader/0.log" Jul 10 19:44:11.722698 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:11.722677 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e879a6-6b80-4809-983b-fcca92cc36f7/kube-rbac-proxy-web/0.log" Jul 10 19:44:11.746995 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:11.746973 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e879a6-6b80-4809-983b-fcca92cc36f7/kube-rbac-proxy/0.log" Jul 10 19:44:11.770117 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:11.770099 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e879a6-6b80-4809-983b-fcca92cc36f7/kube-rbac-proxy-metric/0.log" Jul 10 19:44:11.797657 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:11.797632 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e879a6-6b80-4809-983b-fcca92cc36f7/prom-label-proxy/0.log" Jul 10 19:44:11.822613 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:11.822591 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e879a6-6b80-4809-983b-fcca92cc36f7/init-config-reloader/0.log" Jul 10 19:44:12.068697 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.068625 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-68bb6996cc-xbbz9_f9967697-4322-4e0c-a0ac-5bb77a278420/metrics-server/0.log" Jul 10 19:44:12.091996 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.091970 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-srmfr_7cf3518b-0d8d-4bda-935a-b40a88dc0114/monitoring-plugin/0.log" Jul 10 19:44:12.292634 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.292615 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rp2p5_94b134de-c48b-4194-8cba-0f93ad8481d9/node-exporter/0.log" Jul 10 19:44:12.313315 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.313293 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rp2p5_94b134de-c48b-4194-8cba-0f93ad8481d9/kube-rbac-proxy/0.log" Jul 10 19:44:12.339415 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.339358 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rp2p5_94b134de-c48b-4194-8cba-0f93ad8481d9/init-textfile/0.log" Jul 10 19:44:12.668527 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.668505 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rzjjh_b37b27b3-9815-4488-a231-c024429a6b2e/prometheus-operator/0.log" Jul 10 19:44:12.686174 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.686152 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rzjjh_b37b27b3-9815-4488-a231-c024429a6b2e/kube-rbac-proxy/0.log" Jul 10 19:44:12.843507 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.843479 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6758dff5fd-mcqql_7800a84d-dbef-4e3b-891f-2685ceeb5aa1/thanos-query/0.log" Jul 10 19:44:12.873630 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.873606 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6758dff5fd-mcqql_7800a84d-dbef-4e3b-891f-2685ceeb5aa1/kube-rbac-proxy-web/0.log" Jul 10 19:44:12.910573 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.910543 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6758dff5fd-mcqql_7800a84d-dbef-4e3b-891f-2685ceeb5aa1/kube-rbac-proxy/0.log" Jul 10 19:44:12.942119 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.942050 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6758dff5fd-mcqql_7800a84d-dbef-4e3b-891f-2685ceeb5aa1/prom-label-proxy/0.log" Jul 10 19:44:12.982597 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:12.982568 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6758dff5fd-mcqql_7800a84d-dbef-4e3b-891f-2685ceeb5aa1/kube-rbac-proxy-rules/0.log" Jul 10 19:44:13.022322 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:13.022303 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6758dff5fd-mcqql_7800a84d-dbef-4e3b-891f-2685ceeb5aa1/kube-rbac-proxy-metrics/0.log" Jul 10 19:44:15.086302 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.086269 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-jpj6c_4b00067d-6ef6-44f4-96da-ff149aa4d69c/download-server/0.log" Jul 10 19:44:15.431428 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.431397 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2"] Jul 10 19:44:15.434779 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.434764 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.436976 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.436951 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2vwzv\"/\"kube-root-ca.crt\"" Jul 10 19:44:15.437852 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.437836 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2vwzv\"/\"openshift-service-ca.crt\"" Jul 10 19:44:15.437943 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.437843 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2vwzv\"/\"default-dockercfg-qr9qg\"" Jul 10 19:44:15.444481 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.444460 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2"] Jul 10 19:44:15.576050 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.576019 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnpv4\" (UniqueName: \"kubernetes.io/projected/fdf69647-605b-44bc-b7ad-662b8b5144d9-kube-api-access-gnpv4\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.576205 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.576065 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-sys\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.576205 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.576086 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-proc\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.576205 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.576134 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-podres\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.576315 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.576200 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-lib-modules\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.636828 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.636792 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-k5htc_da9b2366-b93a-4e09-b45b-c974c829e26d/volume-data-source-validator/0.log" Jul 10 19:44:15.676970 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.676948 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-proc\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.677108 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.676980 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-podres\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.677108 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.677017 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-lib-modules\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.677108 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.677032 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnpv4\" (UniqueName: \"kubernetes.io/projected/fdf69647-605b-44bc-b7ad-662b8b5144d9-kube-api-access-gnpv4\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.677108 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.677060 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-sys\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.677108 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.677066 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-proc\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.677277 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.677122 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-sys\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.677277 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.677126 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-podres\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.677277 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.677141 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fdf69647-605b-44bc-b7ad-662b8b5144d9-lib-modules\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.684667 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.684610 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnpv4\" (UniqueName: \"kubernetes.io/projected/fdf69647-605b-44bc-b7ad-662b8b5144d9-kube-api-access-gnpv4\") pod \"perf-node-gather-daemonset-45bf2\" (UID: \"fdf69647-605b-44bc-b7ad-662b8b5144d9\") " pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.744688 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.744623 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:15.872446 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:15.872416 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2"] Jul 10 19:44:15.876862 ip-10-0-137-86 kubenswrapper[2563]: W0710 19:44:15.876831 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfdf69647_605b_44bc_b7ad_662b8b5144d9.slice/crio-0cf0a17cfd59c1075324f775a426d0959336cdea777ba4b057ce25b8d882a620 WatchSource:0}: Error finding container 0cf0a17cfd59c1075324f775a426d0959336cdea777ba4b057ce25b8d882a620: Status 404 returned error can't find the container with id 0cf0a17cfd59c1075324f775a426d0959336cdea777ba4b057ce25b8d882a620 Jul 10 19:44:16.426008 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:16.425974 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" event={"ID":"fdf69647-605b-44bc-b7ad-662b8b5144d9","Type":"ContainerStarted","Data":"afeb0b524889891a1e71e88f8d5631818a1c2adba47a083f7bcb75b6acecfd19"} Jul 10 19:44:16.426706 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:16.426023 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" event={"ID":"fdf69647-605b-44bc-b7ad-662b8b5144d9","Type":"ContainerStarted","Data":"0cf0a17cfd59c1075324f775a426d0959336cdea777ba4b057ce25b8d882a620"} Jul 10 19:44:16.426706 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:16.426100 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:16.443124 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:16.443074 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" podStartSLOduration=1.4430595419999999 podStartE2EDuration="1.443059542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 19:44:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 19:44:16.442162963 +0000 UTC m=+3846.726714269" watchObservedRunningTime="2026-07-10 19:44:16.443059542 +0000 UTC m=+3846.727610850" Jul 10 19:44:16.571198 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:16.571167 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-brwtp_1e89500b-d296-4aa1-afd5-359d27ba0a38/dns/0.log" Jul 10 19:44:16.591686 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:16.591663 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-brwtp_1e89500b-d296-4aa1-afd5-359d27ba0a38/kube-rbac-proxy/0.log" Jul 10 19:44:16.707713 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:16.707599 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4xz7g_cb1e6736-8a0c-4dcd-bef2-a88218277f25/dns-node-resolver/0.log" Jul 10 19:44:17.216999 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:17.216944 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7848468f97-fkr27_469babdc-6a93-4535-9436-5dbb4458405e/registry/0.log" Jul 10 19:44:17.298405 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:17.298379 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xksn6_988b88a2-e62e-4676-91e4-01157f069202/node-ca/0.log" Jul 10 19:44:18.432563 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:18.432535 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w9rnj_9b74d98f-b159-4df8-9a21-79981099cc0e/serve-healthcheck-canary/0.log" Jul 10 19:44:18.845168 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:18.845098 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8wf7f_a220158e-f493-448c-8c20-0b3c640fc0d9/kube-rbac-proxy/0.log" Jul 10 19:44:18.869462 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:18.869439 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8wf7f_a220158e-f493-448c-8c20-0b3c640fc0d9/exporter/0.log" Jul 10 19:44:18.891421 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:18.891399 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8wf7f_a220158e-f493-448c-8c20-0b3c640fc0d9/extractor/0.log" Jul 10 19:44:21.229875 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:21.229844 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-67d95fdc48-rrnhp_957e6b13-0f81-4a4d-a3a6-0e4789beca7e/manager/11.log" Jul 10 19:44:21.230740 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:21.230699 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-67d95fdc48-rrnhp_957e6b13-0f81-4a4d-a3a6-0e4789beca7e/manager/10.log" Jul 10 19:44:21.252202 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:21.252182 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-kdn5r_7c00f2b9-0053-4eeb-b5ba-2d9f022a5cdd/server/0.log" Jul 10 19:44:21.802110 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:21.802081 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-custom-8b44d767d-8bmx9_2f3e8d17-7c37-4c96-b733-1f2ccf5badef/seaweedfs-tls-custom/0.log" Jul 10 19:44:22.440526 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:22.440495 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2vwzv/perf-node-gather-daemonset-45bf2" Jul 10 19:44:24.620694 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:24.620594 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ltklt_4b65b77e-09b9-4937-8014-6431509be7b6/konnectivity-agent/0.log" limit=1024 Jul 10 19:44:25.869534 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:25.869494 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-pbwvk_bb1f8861-ccd7-4b12-bf84-e4722f9349ab/migrator/0.log" Jul 10 19:44:25.890335 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:25.890308 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-pbwvk_bb1f8861-ccd7-4b12-bf84-e4722f9349ab/graceful-termination/0.log" Jul 10 19:44:27.538530 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:27.538501 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xgvw7_7788073c-5d44-49cd-9ae6-51a11ac16848/kube-multus-additional-cni-plugins/0.log" Jul 10 19:44:27.564766 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:27.564745 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xgvw7_7788073c-5d44-49cd-9ae6-51a11ac16848/egress-router-binary-copy/0.log" Jul 10 19:44:27.587552 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:27.587535 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xgvw7_7788073c-5d44-49cd-9ae6-51a11ac16848/cni-plugins/0.log" Jul 10 19:44:27.609479 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:27.609456 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xgvw7_7788073c-5d44-49cd-9ae6-51a11ac16848/bond-cni-plugin/0.log" Jul 10 19:44:27.629896 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:27.629874 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xgvw7_7788073c-5d44-49cd-9ae6-51a11ac16848/routeoverride-cni/0.log" Jul 10 19:44:27.651690 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:27.651673 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xgvw7_7788073c-5d44-49cd-9ae6-51a11ac16848/whereabouts-cni-bincopy/0.log" Jul 10 19:44:27.674516 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:27.674498 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xgvw7_7788073c-5d44-49cd-9ae6-51a11ac16848/whereabouts-cni/0.log" Jul 10 19:44:27.707512 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:27.707491 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-brxbd_4a16c8e1-488a-4082-a279-724ea112e68f/kube-multus/0.log" Jul 10 19:44:27.778816 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:27.778795 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ccjtn_b22d6861-0651-43b0-8642-f1bb3a2c54ee/network-metrics-daemon/0.log" Jul 10 19:44:27.799129 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:27.799072 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ccjtn_b22d6861-0651-43b0-8642-f1bb3a2c54ee/kube-rbac-proxy/0.log" Jul 10 19:44:29.155367 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:29.155339 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-controller/0.log" Jul 10 19:44:29.175437 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:29.175403 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/1.log" Jul 10 19:44:29.175437 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:29.175431 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovn-acl-logging/0.log" Jul 10 19:44:29.199204 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:29.199183 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/kube-rbac-proxy-node/0.log" Jul 10 19:44:29.226454 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:29.226426 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/kube-rbac-proxy-ovn-metrics/0.log" Jul 10 19:44:29.246004 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:29.245982 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/northd/0.log" Jul 10 19:44:29.269051 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:29.269034 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/nbdb/0.log" Jul 10 19:44:29.292015 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:29.291995 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/sbdb/0.log" Jul 10 19:44:29.355568 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:29.355550 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf64p_364b43bc-cd7a-49cb-b214-cdbece4997df/ovnkube-controller/0.log" Jul 10 19:44:30.392158 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:30.392132 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-prjp9_b691ab8e-52aa-40ea-9e2b-ce8d242181ef/network-check-target-container/0.log" Jul 10 19:44:31.322269 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:31.322217 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-8bbcs_944afe1e-fa5a-4b53-903e-25329fec3529/iptables-alerter/0.log" Jul 10 19:44:32.001802 ip-10-0-137-86 kubenswrapper[2563]: I0710 19:44:32.001775 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-jwrjq_c31d3759-3875-4826-bf22-eafe856a5172/tuned/0.log"