Jul 06 20:14:23.032821 ip-10-0-133-106 systemd[1]: Starting Kubernetes Kubelet... Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.431198 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434783 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434794 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434798 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434802 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434805 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434813 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:14:23.462782 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434818 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434822 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434825 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434827 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434830 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434833 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434836 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434839 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434841 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434844 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434847 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434849 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434852 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434855 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434857 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434860 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434863 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434866 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434869 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:14:23.711101 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434871 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:14:23.508023 ip-10-0-133-106 systemd[1]: Started Kubernetes Kubelet. Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434874 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434877 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434879 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434882 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434885 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434887 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434890 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434892 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434895 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434898 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434900 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434903 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434906 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434910 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434913 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434916 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434919 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434922 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:14:23.721887 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434925 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434928 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434931 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434934 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434937 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434940 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434943 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434945 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434948 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434951 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434954 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434956 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434959 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434962 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434965 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434967 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434971 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434976 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434979 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434982 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:14:23.724353 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434984 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434987 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434989 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434992 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434995 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.434998 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435001 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435007 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435010 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435012 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435015 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435018 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435020 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435023 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435025 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435028 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435031 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435033 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435035 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435039 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:14:23.726080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435041 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435044 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435047 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435049 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435052 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435055 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435058 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435060 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435063 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435066 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435069 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435072 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435074 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435077 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435080 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435776 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435784 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435786 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435789 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435792 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:14:23.729163 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435795 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435799 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435801 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435804 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435807 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435810 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435812 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435815 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435818 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435820 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435823 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435825 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435828 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435830 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435833 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435835 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435839 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435843 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435846 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435849 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:14:23.731431 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435854 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435857 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435860 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435863 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435866 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435869 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435872 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435875 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435878 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435880 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435883 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435886 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435889 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435893 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435896 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435899 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435901 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435904 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435907 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435909 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:14:23.733921 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435912 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435915 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435917 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435920 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435922 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435924 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435927 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435933 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435936 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435938 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435941 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435944 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435948 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435950 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435953 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435955 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435958 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435960 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435963 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:14:23.735011 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435966 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435968 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435971 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435973 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435976 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435979 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435981 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435984 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435987 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435989 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435992 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435994 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435996 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.435999 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436001 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436004 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436006 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436008 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436011 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436013 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:14:23.736855 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436017 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436019 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436022 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436024 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436028 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436034 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436036 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436039 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436042 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436046 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436049 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436052 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436055 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436058 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436060 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436409 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436414 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436417 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436419 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:14:23.738080 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436422 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436425 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436428 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436430 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436434 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436438 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436441 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436444 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436447 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436449 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436452 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436454 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436457 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436461 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436463 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436466 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436468 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436470 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436474 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:14:23.739196 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436477 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436479 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436482 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436485 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436487 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436490 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436492 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436495 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436497 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436500 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436502 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436505 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436507 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436510 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436513 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436516 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436518 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436521 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436524 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436526 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:14:23.740003 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436529 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436532 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436534 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436537 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436540 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436542 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436562 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436565 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436567 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436570 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436572 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436576 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436579 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436581 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436584 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436586 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436589 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436591 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436594 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436596 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:14:23.740830 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436599 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436601 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436604 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436606 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436609 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436611 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436614 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436616 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436619 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436621 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436623 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436626 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436628 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436632 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436635 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436638 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436641 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436644 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436648 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:14:23.741786 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436651 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436653 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436656 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436659 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436666 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436670 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436672 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436675 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436677 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436680 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436682 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436685 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436688 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436690 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436693 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436695 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.436697 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436762 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436768 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436775 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436781 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 20:14:23.742386 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436787 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436792 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436798 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436804 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436808 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436811 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436814 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436817 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436820 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436823 2566 flags.go:64] FLAG: --cgroup-root="" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436827 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436830 2566 flags.go:64] FLAG: --client-ca-file="" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436833 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436836 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436839 2566 flags.go:64] FLAG: --cluster-domain="" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436843 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436846 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436849 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436852 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436856 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436859 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436862 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436865 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436868 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 06 20:14:23.743034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436871 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436873 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436877 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436879 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436883 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436886 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436889 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436892 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436894 2566 flags.go:64] FLAG: --enable-server="true" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436898 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436902 2566 flags.go:64] FLAG: --event-burst="100" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436905 2566 flags.go:64] FLAG: --event-qps="50" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436908 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436911 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436914 2566 flags.go:64] FLAG: --eviction-hard="" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436917 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436920 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436923 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436927 2566 flags.go:64] FLAG: --eviction-soft="" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436930 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436932 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436935 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436938 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436942 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436945 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 20:14:23.743668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436948 2566 flags.go:64] FLAG: --feature-gates="" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436951 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436954 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436957 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436960 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436963 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436966 2566 flags.go:64] FLAG: --help="false" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436969 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-133-106.ec2.internal" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436972 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436974 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436977 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436981 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436984 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436986 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436989 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436992 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436994 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.436998 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437001 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437003 2566 flags.go:64] FLAG: --kube-reserved="" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437006 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437009 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437012 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437015 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 20:14:23.744754 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437017 2566 flags.go:64] FLAG: --lock-file="" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437022 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437025 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437027 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437033 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437035 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437039 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437042 2566 flags.go:64] FLAG: --logging-format="text" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437045 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437048 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437051 2566 flags.go:64] FLAG: --manifest-url="" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437053 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437058 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437060 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437064 2566 flags.go:64] FLAG: --max-pods="110" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437067 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437070 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437073 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437076 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437078 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437081 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437084 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437091 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437094 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437097 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 20:14:23.745606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437100 2566 flags.go:64] FLAG: --pod-cidr="" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437103 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437108 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437111 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437114 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437117 2566 flags.go:64] FLAG: --port="10250" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437120 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437122 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f04d7de42f1ce5aa" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437127 2566 flags.go:64] FLAG: --qos-reserved="" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437130 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437132 2566 flags.go:64] FLAG: --register-node="true" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437135 2566 flags.go:64] FLAG: --register-with-taints="" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437138 2566 flags.go:64] FLAG: --registry-burst="10" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437141 2566 flags.go:64] FLAG: --registry-qps="5" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437145 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437148 2566 flags.go:64] FLAG: --reserved-memory="" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437151 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437154 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437157 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437160 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437162 2566 flags.go:64] FLAG: --runonce="false" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437165 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437168 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437171 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437173 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 20:14:23.746301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437176 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437179 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437182 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437185 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437188 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437190 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437193 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437196 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437199 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437202 2566 flags.go:64] FLAG: --system-cgroups="" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437205 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437209 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437212 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437215 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437218 2566 flags.go:64] FLAG: --tls-min-version="" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437222 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437225 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437228 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437231 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437233 2566 flags.go:64] FLAG: --v="2" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437237 2566 flags.go:64] FLAG: --version="false" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437243 2566 flags.go:64] FLAG: --vmodule="" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437247 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437250 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437381 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:14:23.747277 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437385 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437389 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437392 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437395 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437398 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437401 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437403 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437406 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437408 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437411 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437414 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437416 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437419 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437421 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437424 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437426 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437429 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437432 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437434 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437437 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:14:23.748040 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437439 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437442 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437445 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437448 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437450 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437453 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437456 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437458 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437462 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437465 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437467 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437470 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437472 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437475 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437477 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437480 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437483 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437485 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437488 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:14:23.934273 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437490 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437493 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437495 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437498 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437500 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437502 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437505 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437508 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437510 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437512 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437515 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437518 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437520 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437523 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437525 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437530 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437533 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437536 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437539 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:14:23.934883 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437541 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437558 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437563 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437565 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437570 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437573 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437576 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437579 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437582 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437585 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437588 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437591 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437594 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437596 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437599 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437603 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437606 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437609 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437612 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437615 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:14:23.935430 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437618 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437621 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437623 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437626 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437628 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437631 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437634 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437637 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437640 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437643 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437646 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437648 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437651 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437653 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437657 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437659 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437662 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437665 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437668 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:14:23.936023 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.437670 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.437675 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.445771 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.445787 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.445807 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.445813 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445867 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445872 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445876 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445879 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445884 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445888 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445891 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445895 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:14:23.936523 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445897 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445903 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445906 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445909 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445912 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445915 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445917 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445920 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445923 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445925 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445928 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445931 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445933 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445936 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445939 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445941 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445944 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445947 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445951 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:14:23.936975 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445954 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445957 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445960 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445963 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445966 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445968 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445971 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445974 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445976 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445979 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445981 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445984 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445986 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445989 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445994 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445997 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.445999 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446002 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446005 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446008 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:14:23.937476 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446010 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446013 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446015 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446018 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446020 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446023 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446025 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446028 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446030 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446033 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446036 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446038 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446041 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446043 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446046 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446049 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446051 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446054 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446056 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446059 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:14:23.938054 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446062 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446065 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446067 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446070 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446072 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446075 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446079 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446082 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446084 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446086 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446090 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446093 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446097 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446100 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446103 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446106 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446108 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446111 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446114 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446116 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:14:23.938605 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446119 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446121 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446124 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446127 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446129 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446132 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446134 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446137 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446140 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446142 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446145 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446147 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.446152 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446261 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446265 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:14:23.939169 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446268 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446271 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446274 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446276 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446280 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446283 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446286 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446290 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446292 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446295 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446297 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446300 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446302 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446305 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446309 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446313 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446316 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446319 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446321 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:14:23.939598 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446324 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446326 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446329 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446331 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446334 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446337 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446339 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446342 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446345 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446347 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446350 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446352 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446354 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446357 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446359 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446362 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446364 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446368 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446370 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446374 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:14:23.940128 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446379 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446381 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446384 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446387 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446390 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446392 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446396 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446399 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446402 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446405 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446407 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446410 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446412 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446415 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446417 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446420 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446423 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446425 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446428 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:14:23.940670 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446430 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446433 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446436 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446438 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446440 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446443 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446445 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446448 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446450 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446452 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446456 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446459 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446461 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446465 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446467 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446470 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446472 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446475 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446477 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:14:23.941202 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446480 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446482 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446485 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446488 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446490 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446493 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446495 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446498 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446500 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446503 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446505 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446508 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446510 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446527 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446531 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446534 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446537 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446540 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446543 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:14:23.941762 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:23.446567 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.446572 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.446687 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.450635 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.451948 2566 server.go:1019] "Starting client certificate rotation" Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.452051 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.452085 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.455193 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.456374 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.471857 2566 log.go:25] "Validated CRI v1 runtime API" Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.477201 2566 log.go:25] "Validated CRI v1 image API" Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.477992 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 20:14:23.942301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.480258 2566 fs.go:135] Filesystem UUIDs: map[3fa2c565-168b-429e-908c-2580c74b0aef:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a55f03d4-25fb-414a-80e2-848656739a30:/dev/nvme0n1p4] Jul 06 20:14:23.942674 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.480283 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 20:14:23.942674 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.482952 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.485331 2566 manager.go:216] Machine: {Timestamp:2026-07-06 20:14:23.48402632 +0000 UTC m=+0.329891016 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099484 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29700a82a5c5e92d02572be3171e5a SystemUUID:ec29700a-82a5-c5e9-2d02-572be3171e5a BootID:9f175094-fb37-4e2d-8586-5978a2f0426b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d1:50:4c:7b:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d1:50:4c:7b:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:15:22:05:d1:f4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.485423 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.485496 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.488644 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.488668 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-106.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.490050 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.490061 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.490068 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.490078 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.491642 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.494885 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.494997 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.495031 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.495050 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.495065 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.495073 2566 kubelet.go:398] "Adding apiserver pod source" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.495082 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 20:14:23.942883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.495458 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.495476 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.499160 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.501253 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.501282 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502467 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502481 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502489 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502497 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502509 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502517 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502525 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502531 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502537 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502542 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502569 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.502574 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.503352 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.503361 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.507184 2566 server.go:1268] "Started kubelet" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.507261 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.507307 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.507543 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.508486 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.509099 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-106.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.509446 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.509641 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-106.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.509680 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.513246 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.515004 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-106.ec2.internal.18bfcc885cd2bf1b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-106.ec2.internal,UID:ip-10-0-133-106.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-106.ec2.internal,},FirstTimestamp:2026-07-06 20:14:23.506939675 +0000 UTC m=+0.352804371,LastTimestamp:2026-07-06 20:14:23.506939675 +0000 UTC m=+0.352804371,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-106.ec2.internal,}" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.516126 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.516293 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.516322 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.516923 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.516947 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.516970 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.517055 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.517061 2566 factory.go:55] Registering systemd factory Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.517075 2566 factory.go:223] Registration of the systemd container factory successfully Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.517062 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.517268 2566 factory.go:153] Registering CRI-O factory Jul 06 20:14:23.943431 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.517278 2566 factory.go:223] Registration of the crio container factory successfully Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.517321 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.517347 2566 factory.go:103] Registering Raw factory Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.517360 2566 manager.go:1196] Started watching for new ooms in manager Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.518302 2566 manager.go:349] Starting recovery of all containers Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.522198 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.522230 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-106.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.534438 2566 manager.go:354] Recovery completed Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.538932 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.540730 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.540780 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.540794 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.541310 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.541318 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.541322 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.541340 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.542043 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.542073 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.542087 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.542736 2566 policy_none.go:49] "None policy: Start" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.542750 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.542759 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.542945 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-106.ec2.internal.18bfcc885ed6d06d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-106.ec2.internal,UID:ip-10-0-133-106.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-106.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-106.ec2.internal,},FirstTimestamp:2026-07-06 20:14:23.540760685 +0000 UTC m=+0.386625385,LastTimestamp:2026-07-06 20:14:23.540760685 +0000 UTC m=+0.386625385,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-106.ec2.internal,}" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.543854 2566 policy_none.go:47] "Start" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.549186 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t9rjb" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.554839 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-106.ec2.internal.18bfcc885ed7376d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-106.ec2.internal,UID:ip-10-0-133-106.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-106.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-106.ec2.internal,},FirstTimestamp:2026-07-06 20:14:23.540787053 +0000 UTC m=+0.386651749,LastTimestamp:2026-07-06 20:14:23.540787053 +0000 UTC m=+0.386651749,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-106.ec2.internal,}" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.558706 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t9rjb" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.617043 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.660820 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.661971 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.662005 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 20:14:23.944828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.662030 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.662065 2566 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.664851 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.717889 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.753261 2566 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-106.ec2.internal\" not found" node="ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.762338 2566 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.765078 2566 manager.go:337] "Starting Device Plugin manager" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.765161 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.765170 2566 server.go:87] "Starting device plugin registration server" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.765418 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.765428 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.765517 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.765610 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.765619 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.766316 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.766367 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-106.ec2.internal\" not found" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.866343 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.867215 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.867241 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.867251 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.867274 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.880109 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.880129 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-106.ec2.internal\": node \"ip-10-0-133-106.ec2.internal\" not found" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:23.910715 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.962707 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal"] Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.962799 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.964268 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.964293 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.964303 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.965777 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.965893 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.965925 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.966445 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.966449 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.966470 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.966473 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.966481 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.966485 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.967889 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.967908 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.968560 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:14:24.126040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.968583 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:23.968592 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.000309 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-106.ec2.internal\" not found" node="ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.004457 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-106.ec2.internal\" not found" node="ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.010991 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.020361 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8c577c025dd62eaa24f491393f35ac71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"8c577c025dd62eaa24f491393f35ac71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.020381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c577c025dd62eaa24f491393f35ac71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"8c577c025dd62eaa24f491393f35ac71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.020397 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b7cb81d798809d5a2a79e2bc119037e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-106.ec2.internal\" (UID: \"5b7cb81d798809d5a2a79e2bc119037e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.111193 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.121515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8c577c025dd62eaa24f491393f35ac71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"8c577c025dd62eaa24f491393f35ac71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.121537 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c577c025dd62eaa24f491393f35ac71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"8c577c025dd62eaa24f491393f35ac71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.121569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b7cb81d798809d5a2a79e2bc119037e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-106.ec2.internal\" (UID: \"5b7cb81d798809d5a2a79e2bc119037e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.121613 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5b7cb81d798809d5a2a79e2bc119037e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-106.ec2.internal\" (UID: \"5b7cb81d798809d5a2a79e2bc119037e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.121618 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8c577c025dd62eaa24f491393f35ac71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"8c577c025dd62eaa24f491393f35ac71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.128291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.121626 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c577c025dd62eaa24f491393f35ac71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"8c577c025dd62eaa24f491393f35ac71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.312779 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.211678 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found" Jul 06 20:14:24.312779 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.303160 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.312779 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.307250 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.312779 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.312574 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found" Jul 06 20:14:24.459508 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.352367 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:14:24.459508 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.412670 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found" Jul 06 20:14:24.459508 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.451185 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 20:14:24.459508 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.451289 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 20:14:24.459508 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.451326 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.471712 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.495956 2566 apiserver.go:52] "Watching apiserver" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.508657 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.508961 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-hb74l","openshift-multus/multus-q55sw","openshift-network-diagnostics/network-check-target-5slw5","openshift-network-operator/iptables-alerter-tsptr","openshift-ovn-kubernetes/ovnkube-node-cfhzw","kube-system/konnectivity-agent-q2nlz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg","openshift-cluster-node-tuning-operator/tuned-bwgzt","openshift-multus/network-metrics-daemon-r4xhr","kube-system/global-pull-secret-syncer-tjmw4","openshift-image-registry/node-ca-tlhdp"] Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.510798 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.511856 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.514090 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.514107 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-58zgd\"" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.514461 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.514470 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.521187 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.521221 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.521417 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.521640 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.521952 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.521999 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.522060 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zlqph\"" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.522537 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523776 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-system-cni-dir\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523798 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-cni-dir\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523816 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-run-netns\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523832 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-registration-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523858 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-sys-fs\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523874 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-kubernetes\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.523887 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:24.577175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523891 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-sysctl-d\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523931 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-tuned\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523948 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-etc-kubernetes\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523962 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js5l5\" (UniqueName: \"kubernetes.io/projected/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-kube-api-access-js5l5\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.523991 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-sysconfig\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524006 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbv8v\" (UniqueName: \"kubernetes.io/projected/8896ec65-47bb-44a1-8790-6a3883dbf0f5-kube-api-access-qbv8v\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524038 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-cnibin\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524061 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-os-release\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524080 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-modprobe-d\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524098 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-cni-binary-copy\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524113 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-run-k8s-cni-cncf-io\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524128 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-var-lib-cni-multus\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524148 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-var-lib-kubelet\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524185 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-daemon-config\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524203 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-run-multus-certs\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524218 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-host\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.578933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524242 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljc88\" (UniqueName: \"kubernetes.io/projected/87b35021-a783-49e3-ba37-c87948f9f1e9-kube-api-access-ljc88\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524259 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-socket-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524272 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-var-lib-kubelet\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524291 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87b35021-a783-49e3-ba37-c87948f9f1e9-tmp\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524333 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524355 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-device-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524371 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-sysctl-conf\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524389 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-systemd\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-run\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524429 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-sys\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524446 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-lib-modules\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524472 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a44f5a2-9953-4ae9-b0be-769c3f4e0d96-konnectivity-ca\") pod \"konnectivity-agent-q2nlz\" (UID: \"1a44f5a2-9953-4ae9-b0be-769c3f4e0d96\") " pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524492 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-socket-dir-parent\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524507 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-var-lib-cni-bin\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524522 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-hostroot\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524536 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-conf-dir\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.524587 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a44f5a2-9953-4ae9-b0be-769c3f4e0d96-agent-certs\") pod \"konnectivity-agent-q2nlz\" (UID: \"1a44f5a2-9953-4ae9-b0be-769c3f4e0d96\") " pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.525219 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.579784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.526431 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.526485 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.527719 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tsptr" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.529182 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.530611 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.530654 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.531836 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tlhdp" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.535109 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.535248 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.535456 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.539046 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9p58p\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.539380 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.539491 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.539623 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.539766 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540073 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540097 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540158 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540163 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540198 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540209 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540213 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540158 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540161 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540280 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gqpc7\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540303 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540319 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zt57m\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540213 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t8qhr\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540362 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540420 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-82vcn\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540452 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b5xbg\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.540471 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.555528 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 20:14:24.580814 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.559585 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 20:14:24.582347 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.559648 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" Jul 06 20:14:24.582347 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.559967 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal"] Jul 06 20:14:24.582347 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.562014 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 20:09:23 +0000 UTC" deadline="2028-04-23 05:24:09.342834682 +0000 UTC" Jul 06 20:14:24.582347 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.562031 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15753h9m44.780805379s" Jul 06 20:14:24.582347 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.573414 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal"] Jul 06 20:14:24.582347 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.574184 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.618313 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.624819 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-os-release\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.624839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-modprobe-d\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.624857 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1dbb2575-74b1-4231-bfd2-116630ca8579-iptables-alerter-script\") pod \"iptables-alerter-tsptr\" (UID: \"1dbb2575-74b1-4231-bfd2-116630ca8579\") " pod="openshift-network-operator/iptables-alerter-tsptr" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.624873 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c6ffb1c-9153-4f2d-83ba-967ed6a026a1-host\") pod \"node-ca-tlhdp\" (UID: \"1c6ffb1c-9153-4f2d-83ba-967ed6a026a1\") " pod="openshift-image-registry/node-ca-tlhdp" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.624888 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-cni-binary-copy\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.624947 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-os-release\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.624986 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-var-lib-cni-multus\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.624991 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-modprobe-d\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625008 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-var-lib-kubelet\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625023 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljc88\" (UniqueName: \"kubernetes.io/projected/87b35021-a783-49e3-ba37-c87948f9f1e9-kube-api-access-ljc88\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625033 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-var-lib-cni-multus\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625039 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-cnibin\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625058 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-var-lib-openvswitch\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625066 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-var-lib-kubelet\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625075 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-socket-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625090 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-var-lib-kubelet\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.674279 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625104 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87b35021-a783-49e3-ba37-c87948f9f1e9-tmp\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625121 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dxt8\" (UniqueName: \"kubernetes.io/projected/e865c7c9-be01-4556-9120-f629f6a64fc0-kube-api-access-8dxt8\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625136 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12b0e04f-1230-4820-a7ce-93a5ce83ef68-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625154 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec562f06-afde-4298-bd3b-4b00bd4826b5-ovnkube-config\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625168 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-var-lib-kubelet\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625178 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-sysctl-conf\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625208 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-lib-modules\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625257 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-cni-netd\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625282 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-sysctl-conf\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec562f06-afde-4298-bd3b-4b00bd4826b5-ovnkube-script-lib\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625314 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a44f5a2-9953-4ae9-b0be-769c3f4e0d96-konnectivity-ca\") pod \"konnectivity-agent-q2nlz\" (UID: \"1a44f5a2-9953-4ae9-b0be-769c3f4e0d96\") " pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625335 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-socket-dir-parent\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625354 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a44f5a2-9953-4ae9-b0be-769c3f4e0d96-agent-certs\") pod \"konnectivity-agent-q2nlz\" (UID: \"1a44f5a2-9953-4ae9-b0be-769c3f4e0d96\") " pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625375 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c6ffb1c-9153-4f2d-83ba-967ed6a026a1-serviceca\") pod \"node-ca-tlhdp\" (UID: \"1c6ffb1c-9153-4f2d-83ba-967ed6a026a1\") " pod="openshift-image-registry/node-ca-tlhdp" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625396 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-lib-modules\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625398 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-system-cni-dir\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625432 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-run-netns\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.674850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625435 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-socket-dir-parent\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625461 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-run-netns\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625471 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-system-cni-dir\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-sysctl-d\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625517 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-tuned\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625537 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625571 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625588 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-etc-kubernetes\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625603 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js5l5\" (UniqueName: \"kubernetes.io/projected/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-kube-api-access-js5l5\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625614 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-sysctl-d\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625620 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-sysconfig\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625653 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12b0e04f-1230-4820-a7ce-93a5ce83ef68-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625681 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-sysconfig\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625752 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-kubelet\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625770 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-etc-kubernetes\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbv8v\" (UniqueName: \"kubernetes.io/projected/8896ec65-47bb-44a1-8790-6a3883dbf0f5-kube-api-access-qbv8v\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625828 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-node-log\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.675438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625854 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-cnibin\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625877 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-run-ovn\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625902 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec562f06-afde-4298-bd3b-4b00bd4826b5-ovn-node-metrics-cert\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625928 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8wl5\" (UniqueName: \"kubernetes.io/projected/1c6ffb1c-9153-4f2d-83ba-967ed6a026a1-kube-api-access-p8wl5\") pod \"node-ca-tlhdp\" (UID: \"1c6ffb1c-9153-4f2d-83ba-967ed6a026a1\") " pod="openshift-image-registry/node-ca-tlhdp" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625954 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-run-k8s-cni-cncf-io\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.625980 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-daemon-config\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626005 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-run-multus-certs\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626031 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-host\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-system-cni-dir\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626079 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12b0e04f-1230-4820-a7ce-93a5ce83ef68-cni-binary-copy\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626102 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-run-systemd\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626116 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626132 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv82g\" (UniqueName: \"kubernetes.io/projected/12b0e04f-1230-4820-a7ce-93a5ce83ef68-kube-api-access-rv82g\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626148 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/985b8c24-44e1-435e-96c0-e99a601b96c1-kubelet-config\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626165 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twvd9\" (UniqueName: \"kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9\") pod \"network-check-target-5slw5\" (UID: \"812aa8ab-4982-4bba-9655-a5d69680cc8b\") " pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626182 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.675997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626198 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvwtz\" (UniqueName: \"kubernetes.io/projected/ec562f06-afde-4298-bd3b-4b00bd4826b5-kube-api-access-pvwtz\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626216 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626231 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-device-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626246 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-systemd\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626260 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-run\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-sys\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626292 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626306 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-slash\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626322 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-var-lib-cni-bin\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626331 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-cnibin\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626335 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-run-k8s-cni-cncf-io\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626356 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-hostroot\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626373 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-conf-dir\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626383 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-var-lib-cni-bin\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626392 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcr94\" (UniqueName: \"kubernetes.io/projected/1dbb2575-74b1-4231-bfd2-116630ca8579-kube-api-access-hcr94\") pod \"iptables-alerter-tsptr\" (UID: \"1dbb2575-74b1-4231-bfd2-116630ca8579\") " pod="openshift-network-operator/iptables-alerter-tsptr" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626398 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-device-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626410 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-run-netns\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.676482 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626426 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-host-run-multus-certs\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626415 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-sys\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626455 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-etc-openvswitch\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626472 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-log-socket\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626475 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-conf-dir\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626490 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-cni-bin\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626512 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-cni-dir\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626531 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-run\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626536 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-hostroot\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626536 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-registration-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626514 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-systemd\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626585 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626592 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-host\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626619 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-sys-fs\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626646 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-cni-dir\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626668 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-socket-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626692 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-registration-dir\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626695 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-kubernetes\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.676971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626721 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-sys-fs\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-os-release\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626778 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/985b8c24-44e1-435e-96c0-e99a601b96c1-dbus\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626783 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-kubernetes\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626807 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec562f06-afde-4298-bd3b-4b00bd4826b5-env-overrides\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626838 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626867 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1dbb2575-74b1-4231-bfd2-116630ca8579-host-slash\") pod \"iptables-alerter-tsptr\" (UID: \"1dbb2575-74b1-4231-bfd2-116630ca8579\") " pod="openshift-network-operator/iptables-alerter-tsptr" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626883 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8896ec65-47bb-44a1-8790-6a3883dbf0f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626895 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-systemd-units\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.626919 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-run-openvswitch\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.627251 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-cni-binary-copy\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.627297 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-multus-daemon-config\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.628410 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a44f5a2-9953-4ae9-b0be-769c3f4e0d96-konnectivity-ca\") pod \"konnectivity-agent-q2nlz\" (UID: \"1a44f5a2-9953-4ae9-b0be-769c3f4e0d96\") " pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.629676 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87b35021-a783-49e3-ba37-c87948f9f1e9-etc-tuned\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.629722 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87b35021-a783-49e3-ba37-c87948f9f1e9-tmp\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.629912 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a44f5a2-9953-4ae9-b0be-769c3f4e0d96-agent-certs\") pod \"konnectivity-agent-q2nlz\" (UID: \"1a44f5a2-9953-4ae9-b0be-769c3f4e0d96\") " pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.637342 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:14:24.677626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.642738 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js5l5\" (UniqueName: \"kubernetes.io/projected/19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8-kube-api-access-js5l5\") pod \"multus-q55sw\" (UID: \"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8\") " pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.678110 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.642821 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbv8v\" (UniqueName: \"kubernetes.io/projected/8896ec65-47bb-44a1-8790-6a3883dbf0f5-kube-api-access-qbv8v\") pod \"aws-ebs-csi-driver-node-q2kbg\" (UID: \"8896ec65-47bb-44a1-8790-6a3883dbf0f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.678110 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.642843 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljc88\" (UniqueName: \"kubernetes.io/projected/87b35021-a783-49e3-ba37-c87948f9f1e9-kube-api-access-ljc88\") pod \"tuned-bwgzt\" (UID: \"87b35021-a783-49e3-ba37-c87948f9f1e9\") " pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727661 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1dbb2575-74b1-4231-bfd2-116630ca8579-iptables-alerter-script\") pod \"iptables-alerter-tsptr\" (UID: \"1dbb2575-74b1-4231-bfd2-116630ca8579\") " pod="openshift-network-operator/iptables-alerter-tsptr" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727680 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c6ffb1c-9153-4f2d-83ba-967ed6a026a1-host\") pod \"node-ca-tlhdp\" (UID: \"1c6ffb1c-9153-4f2d-83ba-967ed6a026a1\") " pod="openshift-image-registry/node-ca-tlhdp" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727703 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-cnibin\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727718 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-var-lib-openvswitch\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727734 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dxt8\" (UniqueName: \"kubernetes.io/projected/e865c7c9-be01-4556-9120-f629f6a64fc0-kube-api-access-8dxt8\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727748 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12b0e04f-1230-4820-a7ce-93a5ce83ef68-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727763 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec562f06-afde-4298-bd3b-4b00bd4826b5-ovnkube-config\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727776 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-cni-netd\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727789 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec562f06-afde-4298-bd3b-4b00bd4826b5-ovnkube-script-lib\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727802 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c6ffb1c-9153-4f2d-83ba-967ed6a026a1-serviceca\") pod \"node-ca-tlhdp\" (UID: \"1c6ffb1c-9153-4f2d-83ba-967ed6a026a1\") " pod="openshift-image-registry/node-ca-tlhdp" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727830 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727845 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12b0e04f-1230-4820-a7ce-93a5ce83ef68-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727867 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-kubelet\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727880 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-node-log\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727893 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-run-ovn\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816169 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec562f06-afde-4298-bd3b-4b00bd4826b5-ovn-node-metrics-cert\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727923 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8wl5\" (UniqueName: \"kubernetes.io/projected/1c6ffb1c-9153-4f2d-83ba-967ed6a026a1-kube-api-access-p8wl5\") pod \"node-ca-tlhdp\" (UID: \"1c6ffb1c-9153-4f2d-83ba-967ed6a026a1\") " pod="openshift-image-registry/node-ca-tlhdp" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727938 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-system-cni-dir\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727952 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12b0e04f-1230-4820-a7ce-93a5ce83ef68-cni-binary-copy\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727966 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-run-systemd\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727981 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.727994 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv82g\" (UniqueName: \"kubernetes.io/projected/12b0e04f-1230-4820-a7ce-93a5ce83ef68-kube-api-access-rv82g\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728007 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/985b8c24-44e1-435e-96c0-e99a601b96c1-kubelet-config\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvd9\" (UniqueName: \"kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9\") pod \"network-check-target-5slw5\" (UID: \"812aa8ab-4982-4bba-9655-a5d69680cc8b\") " pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728040 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728054 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvwtz\" (UniqueName: \"kubernetes.io/projected/ec562f06-afde-4298-bd3b-4b00bd4826b5-kube-api-access-pvwtz\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728070 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728083 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-slash\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcr94\" (UniqueName: \"kubernetes.io/projected/1dbb2575-74b1-4231-bfd2-116630ca8579-kube-api-access-hcr94\") pod \"iptables-alerter-tsptr\" (UID: \"1dbb2575-74b1-4231-bfd2-116630ca8579\") " pod="openshift-network-operator/iptables-alerter-tsptr" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728111 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-run-netns\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728123 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-etc-openvswitch\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-log-socket\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.816882 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728152 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-cni-bin\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728167 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-os-release\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728179 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/985b8c24-44e1-435e-96c0-e99a601b96c1-dbus\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728191 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec562f06-afde-4298-bd3b-4b00bd4826b5-env-overrides\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728205 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1dbb2575-74b1-4231-bfd2-116630ca8579-host-slash\") pod \"iptables-alerter-tsptr\" (UID: \"1dbb2575-74b1-4231-bfd2-116630ca8579\") " pod="openshift-network-operator/iptables-alerter-tsptr" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-systemd-units\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728232 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-run-openvswitch\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728280 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-run-openvswitch\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728642 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1dbb2575-74b1-4231-bfd2-116630ca8579-iptables-alerter-script\") pod \"iptables-alerter-tsptr\" (UID: \"1dbb2575-74b1-4231-bfd2-116630ca8579\") " pod="openshift-network-operator/iptables-alerter-tsptr" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.728681 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c6ffb1c-9153-4f2d-83ba-967ed6a026a1-host\") pod \"node-ca-tlhdp\" (UID: \"1c6ffb1c-9153-4f2d-83ba-967ed6a026a1\") " pod="openshift-image-registry/node-ca-tlhdp" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.729695 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-run-systemd\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.729743 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-run-netns\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.729815 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.729847 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/985b8c24-44e1-435e-96c0-e99a601b96c1-kubelet-config\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.729906 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-node-log\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.729907 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-var-lib-openvswitch\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.729937 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.817523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.729955 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-cni-bin\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.730036 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs podName:e865c7c9-be01-4556-9120-f629f6a64fc0 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:25.229993085 +0000 UTC m=+2.075857788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs") pod "network-metrics-daemon-r4xhr" (UID: "e865c7c9-be01-4556-9120-f629f6a64fc0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730047 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-slash\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730115 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-os-release\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730153 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-system-cni-dir\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730285 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730346 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1dbb2575-74b1-4231-bfd2-116630ca8579-host-slash\") pod \"iptables-alerter-tsptr\" (UID: \"1dbb2575-74b1-4231-bfd2-116630ca8579\") " pod="openshift-network-operator/iptables-alerter-tsptr" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.729819 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-log-socket\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730408 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12b0e04f-1230-4820-a7ce-93a5ce83ef68-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730434 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/985b8c24-44e1-435e-96c0-e99a601b96c1-dbus\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730413 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-run-ovn\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730501 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-etc-openvswitch\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730501 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-systemd-units\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730557 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730583 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12b0e04f-1230-4820-a7ce-93a5ce83ef68-cnibin\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730626 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-kubelet\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.730656 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:24.818109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730705 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec562f06-afde-4298-bd3b-4b00bd4826b5-host-cni-netd\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.730715 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret podName:985b8c24-44e1-435e-96c0-e99a601b96c1 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:25.230699495 +0000 UTC m=+2.076564191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret") pod "global-pull-secret-syncer-tjmw4" (UID: "985b8c24-44e1-435e-96c0-e99a601b96c1") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.730657 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec562f06-afde-4298-bd3b-4b00bd4826b5-env-overrides\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.731049 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12b0e04f-1230-4820-a7ce-93a5ce83ef68-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.731112 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c6ffb1c-9153-4f2d-83ba-967ed6a026a1-serviceca\") pod \"node-ca-tlhdp\" (UID: \"1c6ffb1c-9153-4f2d-83ba-967ed6a026a1\") " pod="openshift-image-registry/node-ca-tlhdp" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.731245 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec562f06-afde-4298-bd3b-4b00bd4826b5-ovnkube-config\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.731265 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec562f06-afde-4298-bd3b-4b00bd4826b5-ovnkube-script-lib\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.731296 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12b0e04f-1230-4820-a7ce-93a5ce83ef68-cni-binary-copy\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.733120 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec562f06-afde-4298-bd3b-4b00bd4826b5-ovn-node-metrics-cert\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.748225 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-492n7" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.752855 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.752866 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.752875 2566 projected.go:196] Error preparing data for projected volume kube-api-access-twvd9 for pod openshift-network-diagnostics/network-check-target-5slw5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:24.752920 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9 podName:812aa8ab-4982-4bba-9655-a5d69680cc8b nodeName:}" failed. No retries permitted until 2026-07-06 20:14:25.25291006 +0000 UTC m=+2.098774754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-twvd9" (UniqueName: "kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9") pod "network-check-target-5slw5" (UID: "812aa8ab-4982-4bba-9655-a5d69680cc8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.754579 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcr94\" (UniqueName: \"kubernetes.io/projected/1dbb2575-74b1-4231-bfd2-116630ca8579-kube-api-access-hcr94\") pod \"iptables-alerter-tsptr\" (UID: \"1dbb2575-74b1-4231-bfd2-116630ca8579\") " pod="openshift-network-operator/iptables-alerter-tsptr" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.754993 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8wl5\" (UniqueName: \"kubernetes.io/projected/1c6ffb1c-9153-4f2d-83ba-967ed6a026a1-kube-api-access-p8wl5\") pod \"node-ca-tlhdp\" (UID: \"1c6ffb1c-9153-4f2d-83ba-967ed6a026a1\") " pod="openshift-image-registry/node-ca-tlhdp" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.759745 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvwtz\" (UniqueName: \"kubernetes.io/projected/ec562f06-afde-4298-bd3b-4b00bd4826b5-kube-api-access-pvwtz\") pod \"ovnkube-node-cfhzw\" (UID: \"ec562f06-afde-4298-bd3b-4b00bd4826b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:24.818761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.760498 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dxt8\" (UniqueName: \"kubernetes.io/projected/e865c7c9-be01-4556-9120-f629f6a64fc0-kube-api-access-8dxt8\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:24.819285 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.761883 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv82g\" (UniqueName: \"kubernetes.io/projected/12b0e04f-1230-4820-a7ce-93a5ce83ef68-kube-api-access-rv82g\") pod \"multus-additional-cni-plugins-hb74l\" (UID: \"12b0e04f-1230-4820-a7ce-93a5ce83ef68\") " pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:24.819285 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.764195 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-492n7" Jul 06 20:14:24.951398 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.845498 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q55sw" Jul 06 20:14:24.951398 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.867286 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:24.951398 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.909329 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" Jul 06 20:14:24.951398 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:24.926215 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7cb81d798809d5a2a79e2bc119037e.slice/crio-a70500154c206e2cc3c3e7fd73bd62378e0691d392d131b14d75f02cf27856af WatchSource:0}: Error finding container a70500154c206e2cc3c3e7fd73bd62378e0691d392d131b14d75f02cf27856af: Status 404 returned error can't find the container with id a70500154c206e2cc3c3e7fd73bd62378e0691d392d131b14d75f02cf27856af Jul 06 20:14:24.951398 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:24.926571 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c577c025dd62eaa24f491393f35ac71.slice/crio-70e7821a6498958a839761b47e4822871afa284ba05ca23627535ba3cfffb6eb WatchSource:0}: Error finding container 70e7821a6498958a839761b47e4822871afa284ba05ca23627535ba3cfffb6eb: Status 404 returned error can't find the container with id 70e7821a6498958a839761b47e4822871afa284ba05ca23627535ba3cfffb6eb Jul 06 20:14:24.951398 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:24.927403 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8896ec65_47bb_44a1_8790_6a3883dbf0f5.slice/crio-a8c172668fc142549a7f6a987f610649bba7645782563f7d6934a4c7404e76ab WatchSource:0}: Error finding container a8c172668fc142549a7f6a987f610649bba7645782563f7d6934a4c7404e76ab: Status 404 returned error can't find the container with id a8c172668fc142549a7f6a987f610649bba7645782563f7d6934a4c7404e76ab Jul 06 20:14:24.951398 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.934647 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:14:24.951398 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.938251 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" Jul 06 20:14:24.951398 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:24.947840 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87b35021_a783_49e3_ba37_c87948f9f1e9.slice/crio-2d7355c7376899ff2e03d8184a08b3fdf46e75ab555aea3e35b5d093d13ab620 WatchSource:0}: Error finding container 2d7355c7376899ff2e03d8184a08b3fdf46e75ab555aea3e35b5d093d13ab620: Status 404 returned error can't find the container with id 2d7355c7376899ff2e03d8184a08b3fdf46e75ab555aea3e35b5d093d13ab620 Jul 06 20:14:25.089757 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.962074 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hb74l" Jul 06 20:14:25.089757 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:24.969641 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12b0e04f_1230_4820_a7ce_93a5ce83ef68.slice/crio-e3a53eee850d8aef985e593cfed908c7e37776b36452ba7ec731fa62107b8094 WatchSource:0}: Error finding container e3a53eee850d8aef985e593cfed908c7e37776b36452ba7ec731fa62107b8094: Status 404 returned error can't find the container with id e3a53eee850d8aef985e593cfed908c7e37776b36452ba7ec731fa62107b8094 Jul 06 20:14:25.089757 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:24.974356 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tsptr" Jul 06 20:14:25.089757 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:24.980409 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dbb2575_74b1_4231_bfd2_116630ca8579.slice/crio-daf158bbaf758618b0168fe0baa2413333a8784a8181092b8a8618ad0357797d WatchSource:0}: Error finding container daf158bbaf758618b0168fe0baa2413333a8784a8181092b8a8618ad0357797d: Status 404 returned error can't find the container with id daf158bbaf758618b0168fe0baa2413333a8784a8181092b8a8618ad0357797d Jul 06 20:14:25.089757 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.017687 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:25.089757 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.021779 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tlhdp" Jul 06 20:14:25.089757 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:25.030085 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec562f06_afde_4298_bd3b_4b00bd4826b5.slice/crio-eae9613924dddc57b4497dd19e4f53d77b7e561fa8915d137824f4f0f319a95e WatchSource:0}: Error finding container eae9613924dddc57b4497dd19e4f53d77b7e561fa8915d137824f4f0f319a95e: Status 404 returned error can't find the container with id eae9613924dddc57b4497dd19e4f53d77b7e561fa8915d137824f4f0f319a95e Jul 06 20:14:25.089757 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:25.031757 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6ffb1c_9153_4f2d_83ba_967ed6a026a1.slice/crio-55cc72bdd0deb2979df2d88f47c68426750bdd232447ec74bb833314931d2ede WatchSource:0}: Error finding container 55cc72bdd0deb2979df2d88f47c68426750bdd232447ec74bb833314931d2ede: Status 404 returned error can't find the container with id 55cc72bdd0deb2979df2d88f47c68426750bdd232447ec74bb833314931d2ede Jul 06 20:14:25.237778 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:25.142586 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19dcbafe_60f4_40a5_9ff5_a5c7b48f3ce8.slice/crio-597159108d35ff9d2637f354058bf3b6b30120548966537b55d454af8cfb659b WatchSource:0}: Error finding container 597159108d35ff9d2637f354058bf3b6b30120548966537b55d454af8cfb659b: Status 404 returned error can't find the container with id 597159108d35ff9d2637f354058bf3b6b30120548966537b55d454af8cfb659b Jul 06 20:14:25.237778 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.230534 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:25.237778 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:25.230655 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:25.237778 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:25.230708 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs podName:e865c7c9-be01-4556-9120-f629f6a64fc0 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:26.230695275 +0000 UTC m=+3.076559955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs") pod "network-metrics-daemon-r4xhr" (UID: "e865c7c9-be01-4556-9120-f629f6a64fc0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:25.398795 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:25.286052 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a44f5a2_9953_4ae9_b0be_769c3f4e0d96.slice/crio-b254b290cab3cb6ba6113e8f7bbbfab8eb06222ccdf86d64d481542a079625e8 WatchSource:0}: Error finding container b254b290cab3cb6ba6113e8f7bbbfab8eb06222ccdf86d64d481542a079625e8: Status 404 returned error can't find the container with id b254b290cab3cb6ba6113e8f7bbbfab8eb06222ccdf86d64d481542a079625e8 Jul 06 20:14:25.398795 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.331605 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:25.398795 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.331656 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvd9\" (UniqueName: \"kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9\") pod \"network-check-target-5slw5\" (UID: \"812aa8ab-4982-4bba-9655-a5d69680cc8b\") " pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:25.398795 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:25.331747 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:25.398795 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:25.331760 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:14:25.398795 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:25.331774 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:14:25.398795 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:25.331785 2566 projected.go:196] Error preparing data for projected volume kube-api-access-twvd9 for pod openshift-network-diagnostics/network-check-target-5slw5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:25.398795 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:25.331799 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret podName:985b8c24-44e1-435e-96c0-e99a601b96c1 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:26.331784864 +0000 UTC m=+3.177649545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret") pod "global-pull-secret-syncer-tjmw4" (UID: "985b8c24-44e1-435e-96c0-e99a601b96c1") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:25.398795 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:25.331822 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9 podName:812aa8ab-4982-4bba-9655-a5d69680cc8b nodeName:}" failed. No retries permitted until 2026-07-06 20:14:26.331809341 +0000 UTC m=+3.177674046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-twvd9" (UniqueName: "kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9") pod "network-check-target-5slw5" (UID: "812aa8ab-4982-4bba-9655-a5d69680cc8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:25.665101 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.664045 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:25.665101 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:25.664161 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:25.665101 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.664461 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:25.665101 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:25.664562 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:25.665101 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.664610 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:25.665101 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:25.664676 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:25.728195 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.674105 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" event={"ID":"8c577c025dd62eaa24f491393f35ac71","Type":"ContainerStarted","Data":"70e7821a6498958a839761b47e4822871afa284ba05ca23627535ba3cfffb6eb"} Jul 06 20:14:25.728195 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.679065 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q2nlz" event={"ID":"1a44f5a2-9953-4ae9-b0be-769c3f4e0d96","Type":"ContainerStarted","Data":"b254b290cab3cb6ba6113e8f7bbbfab8eb06222ccdf86d64d481542a079625e8"} Jul 06 20:14:25.728195 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.680757 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q55sw" event={"ID":"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8","Type":"ContainerStarted","Data":"597159108d35ff9d2637f354058bf3b6b30120548966537b55d454af8cfb659b"} Jul 06 20:14:25.728195 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.683327 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" event={"ID":"ec562f06-afde-4298-bd3b-4b00bd4826b5","Type":"ContainerStarted","Data":"eae9613924dddc57b4497dd19e4f53d77b7e561fa8915d137824f4f0f319a95e"} Jul 06 20:14:25.728195 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.685803 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" event={"ID":"87b35021-a783-49e3-ba37-c87948f9f1e9","Type":"ContainerStarted","Data":"2d7355c7376899ff2e03d8184a08b3fdf46e75ab555aea3e35b5d093d13ab620"} Jul 06 20:14:25.728195 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.688441 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tlhdp" event={"ID":"1c6ffb1c-9153-4f2d-83ba-967ed6a026a1","Type":"ContainerStarted","Data":"55cc72bdd0deb2979df2d88f47c68426750bdd232447ec74bb833314931d2ede"} Jul 06 20:14:25.728195 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.690033 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tsptr" event={"ID":"1dbb2575-74b1-4231-bfd2-116630ca8579","Type":"ContainerStarted","Data":"daf158bbaf758618b0168fe0baa2413333a8784a8181092b8a8618ad0357797d"} Jul 06 20:14:25.728195 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.693591 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb74l" event={"ID":"12b0e04f-1230-4820-a7ce-93a5ce83ef68","Type":"ContainerStarted","Data":"e3a53eee850d8aef985e593cfed908c7e37776b36452ba7ec731fa62107b8094"} Jul 06 20:14:25.728195 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.696246 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" event={"ID":"5b7cb81d798809d5a2a79e2bc119037e","Type":"ContainerStarted","Data":"a70500154c206e2cc3c3e7fd73bd62378e0691d392d131b14d75f02cf27856af"} Jul 06 20:14:25.728195 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.700120 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" event={"ID":"8896ec65-47bb-44a1-8790-6a3883dbf0f5","Type":"ContainerStarted","Data":"a8c172668fc142549a7f6a987f610649bba7645782563f7d6934a4c7404e76ab"} Jul 06 20:14:25.768348 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.767317 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 20:09:24 +0000 UTC" deadline="2028-04-19 02:22:53.506308428 +0000 UTC" Jul 06 20:14:25.768348 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:25.767356 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15654h8m27.738956637s" Jul 06 20:14:26.026621 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:26.026586 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:14:26.238910 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:26.238257 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:26.238910 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:26.238451 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:26.238910 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:26.238526 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs podName:e865c7c9-be01-4556-9120-f629f6a64fc0 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:28.238507786 +0000 UTC m=+5.084372487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs") pod "network-metrics-daemon-r4xhr" (UID: "e865c7c9-be01-4556-9120-f629f6a64fc0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:26.338953 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:26.338870 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvd9\" (UniqueName: \"kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9\") pod \"network-check-target-5slw5\" (UID: \"812aa8ab-4982-4bba-9655-a5d69680cc8b\") " pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:26.339117 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:26.338956 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:26.339117 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:26.339086 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:26.339235 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:26.339147 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret podName:985b8c24-44e1-435e-96c0-e99a601b96c1 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:28.339129124 +0000 UTC m=+5.184993830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret") pod "global-pull-secret-syncer-tjmw4" (UID: "985b8c24-44e1-435e-96c0-e99a601b96c1") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:26.339579 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:26.339539 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:14:26.339671 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:26.339581 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:14:26.339671 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:26.339595 2566 projected.go:196] Error preparing data for projected volume kube-api-access-twvd9 for pod openshift-network-diagnostics/network-check-target-5slw5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:26.339671 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:26.339642 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9 podName:812aa8ab-4982-4bba-9655-a5d69680cc8b nodeName:}" failed. No retries permitted until 2026-07-06 20:14:28.339627772 +0000 UTC m=+5.185492474 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-twvd9" (UniqueName: "kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9") pod "network-check-target-5slw5" (UID: "812aa8ab-4982-4bba-9655-a5d69680cc8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:26.722753 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:26.722486 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:14:26.768610 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:26.768517 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 20:09:24 +0000 UTC" deadline="2028-02-26 07:00:01.712152242 +0000 UTC" Jul 06 20:14:26.768610 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:26.768569 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14386h45m34.943587679s" Jul 06 20:14:27.665932 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:27.665033 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:27.665932 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:27.665176 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:27.665932 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:27.665614 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:27.665932 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:27.665727 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:27.665932 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:27.665616 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:27.665932 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:27.665825 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:28.257338 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:28.257296 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:28.257912 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:28.257457 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:28.257912 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:28.257530 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs podName:e865c7c9-be01-4556-9120-f629f6a64fc0 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:32.257514426 +0000 UTC m=+9.103379124 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs") pod "network-metrics-daemon-r4xhr" (UID: "e865c7c9-be01-4556-9120-f629f6a64fc0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:28.358503 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:28.358463 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvd9\" (UniqueName: \"kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9\") pod \"network-check-target-5slw5\" (UID: \"812aa8ab-4982-4bba-9655-a5d69680cc8b\") " pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:28.358674 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:28.358556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:28.358733 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:28.358719 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:28.358800 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:28.358778 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret podName:985b8c24-44e1-435e-96c0-e99a601b96c1 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:32.358760229 +0000 UTC m=+9.204624931 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret") pod "global-pull-secret-syncer-tjmw4" (UID: "985b8c24-44e1-435e-96c0-e99a601b96c1") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:28.359210 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:28.359194 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:14:28.359277 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:28.359218 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:14:28.359277 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:28.359232 2566 projected.go:196] Error preparing data for projected volume kube-api-access-twvd9 for pod openshift-network-diagnostics/network-check-target-5slw5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:28.359380 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:28.359276 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9 podName:812aa8ab-4982-4bba-9655-a5d69680cc8b nodeName:}" failed. No retries permitted until 2026-07-06 20:14:32.359261697 +0000 UTC m=+9.205126401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-twvd9" (UniqueName: "kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9") pod "network-check-target-5slw5" (UID: "812aa8ab-4982-4bba-9655-a5d69680cc8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:29.663179 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:29.662579 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:29.663179 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:29.662579 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:29.663179 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:29.662852 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:29.663179 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:29.662718 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:29.663179 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:29.662911 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:29.663179 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:29.662980 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:31.664150 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:31.663666 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:31.664150 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:31.663803 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:31.664150 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:31.663900 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:31.664150 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:31.663941 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:31.664150 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:31.664037 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:31.664150 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:31.664114 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:32.288220 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:32.288179 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:32.288374 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:32.288326 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:32.288438 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:32.288390 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs podName:e865c7c9-be01-4556-9120-f629f6a64fc0 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:40.288373684 +0000 UTC m=+17.134238383 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs") pod "network-metrics-daemon-r4xhr" (UID: "e865c7c9-be01-4556-9120-f629f6a64fc0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:32.388524 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:32.388486 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:32.388690 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:32.388569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvd9\" (UniqueName: \"kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9\") pod \"network-check-target-5slw5\" (UID: \"812aa8ab-4982-4bba-9655-a5d69680cc8b\") " pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:32.388690 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:32.388661 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:32.388690 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:32.388687 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:14:32.388838 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:32.388703 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:14:32.388838 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:32.388712 2566 projected.go:196] Error preparing data for projected volume kube-api-access-twvd9 for pod openshift-network-diagnostics/network-check-target-5slw5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:32.388838 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:32.388740 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret podName:985b8c24-44e1-435e-96c0-e99a601b96c1 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:40.388719713 +0000 UTC m=+17.234584414 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret") pod "global-pull-secret-syncer-tjmw4" (UID: "985b8c24-44e1-435e-96c0-e99a601b96c1") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:32.388838 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:32.388764 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9 podName:812aa8ab-4982-4bba-9655-a5d69680cc8b nodeName:}" failed. No retries permitted until 2026-07-06 20:14:40.38875438 +0000 UTC m=+17.234619064 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-twvd9" (UniqueName: "kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9") pod "network-check-target-5slw5" (UID: "812aa8ab-4982-4bba-9655-a5d69680cc8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:33.669099 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:33.668328 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:33.669099 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:33.668458 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:33.669099 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:33.668832 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:33.669099 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:33.668921 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:33.669099 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:33.668980 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:33.669099 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:33.669048 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:35.662993 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:35.662960 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:35.663465 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:35.663103 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:35.663535 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:35.663500 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:35.663645 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:35.663608 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:35.663700 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:35.663674 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:35.663773 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:35.663743 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:37.663286 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:37.663216 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:37.663286 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:37.663232 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:37.663286 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:37.663216 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:37.663867 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:37.663351 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:37.663867 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:37.663485 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:37.663867 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:37.663599 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:39.662830 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:39.662801 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:39.662830 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:39.662815 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:39.663356 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:39.662942 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:39.663356 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:39.663019 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:39.663356 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:39.663079 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:39.663356 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:39.663161 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:40.343098 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:40.343066 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:40.343252 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:40.343213 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:40.343313 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:40.343283 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs podName:e865c7c9-be01-4556-9120-f629f6a64fc0 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:56.343267109 +0000 UTC m=+33.189131792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs") pod "network-metrics-daemon-r4xhr" (UID: "e865c7c9-be01-4556-9120-f629f6a64fc0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:40.444114 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:40.444077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvd9\" (UniqueName: \"kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9\") pod \"network-check-target-5slw5\" (UID: \"812aa8ab-4982-4bba-9655-a5d69680cc8b\") " pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:40.444257 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:40.444163 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:40.444257 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:40.444249 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:14:40.444374 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:40.444271 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:14:40.444374 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:40.444272 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:40.444374 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:40.444280 2566 projected.go:196] Error preparing data for projected volume kube-api-access-twvd9 for pod openshift-network-diagnostics/network-check-target-5slw5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:40.444374 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:40.444335 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret podName:985b8c24-44e1-435e-96c0-e99a601b96c1 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:56.444315038 +0000 UTC m=+33.290179738 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret") pod "global-pull-secret-syncer-tjmw4" (UID: "985b8c24-44e1-435e-96c0-e99a601b96c1") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:40.444374 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:40.444354 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9 podName:812aa8ab-4982-4bba-9655-a5d69680cc8b nodeName:}" failed. No retries permitted until 2026-07-06 20:14:56.444344171 +0000 UTC m=+33.290208858 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-twvd9" (UniqueName: "kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9") pod "network-check-target-5slw5" (UID: "812aa8ab-4982-4bba-9655-a5d69680cc8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:41.662980 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:41.662950 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:41.663515 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:41.662997 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:41.663515 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:41.662956 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:41.663515 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:41.663074 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:41.663515 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:41.663149 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:41.663515 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:41.663235 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:43.663369 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:43.663335 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:43.663861 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:43.663441 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:43.663861 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:43.663520 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:43.663861 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:43.663637 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:43.663861 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:43.663679 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:43.663861 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:43.663784 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:45.663129 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.662958 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:45.663590 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.663012 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:45.663590 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:45.663241 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:45.663590 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.662958 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:45.663590 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:45.663321 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:45.663590 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:45.663393 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:45.748794 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.748760 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q55sw" event={"ID":"19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8","Type":"ContainerStarted","Data":"231268732cf8a01f8549df5dce9e1bb38668478912f20bb16316e729050018f8"} Jul 06 20:14:45.752682 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.752647 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" event={"ID":"ec562f06-afde-4298-bd3b-4b00bd4826b5","Type":"ContainerStarted","Data":"9c49170c618fa77af641b0e438694ab11239a7cd1fba5383db9e2583e5850bb3"} Jul 06 20:14:45.752792 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.752713 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" event={"ID":"ec562f06-afde-4298-bd3b-4b00bd4826b5","Type":"ContainerStarted","Data":"2de91612a4ced33e57861d76780dc742f943d259fc2620b77f5f281919d4b594"} Jul 06 20:14:45.752792 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.752760 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" event={"ID":"ec562f06-afde-4298-bd3b-4b00bd4826b5","Type":"ContainerStarted","Data":"b1b717f36c072999f1a116b427095dffb3b3325c2b71c064d215a4e7a6ba8a0d"} Jul 06 20:14:45.752883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.752801 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" event={"ID":"ec562f06-afde-4298-bd3b-4b00bd4826b5","Type":"ContainerStarted","Data":"930d4c05b42a7995bd1ee0b4e078aa9c8222fe8ec00345e5f5dcfe44278dabae"} Jul 06 20:14:45.752883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.752838 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" event={"ID":"ec562f06-afde-4298-bd3b-4b00bd4826b5","Type":"ContainerStarted","Data":"f15e1bcd2ec4b2008871949942532f454bc17346d3909e1542af2c27281500a6"} Jul 06 20:14:45.752883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.752880 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" event={"ID":"ec562f06-afde-4298-bd3b-4b00bd4826b5","Type":"ContainerStarted","Data":"10f04f217cbf1364dbb9ee8f02c8a7b294b1c37aaf93fad17f58431560079de9"} Jul 06 20:14:45.754127 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.754102 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" event={"ID":"87b35021-a783-49e3-ba37-c87948f9f1e9","Type":"ContainerStarted","Data":"8708f3181db28d5f6ca33fc26305bde7cc4df87182d56906d1628ab5967b97a0"} Jul 06 20:14:45.755724 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.755698 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" event={"ID":"5b7cb81d798809d5a2a79e2bc119037e","Type":"ContainerStarted","Data":"eebed0b92c4f33adeb579ac2a01bc62eb1c17bc3341fa3ddab96147f8fd243f5"} Jul 06 20:14:45.791517 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.791474 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q55sw" podStartSLOduration=3.103179649 podStartE2EDuration="22.791462936s" totalImagesPullingTime="19.688283287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:45.790424671 +0000 UTC m=+22.636289375" watchObservedRunningTime="2026-07-06 20:14:45.791462936 +0000 UTC m=+22.637327641" Jul 06 20:14:45.814070 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.813997 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bwgzt" podStartSLOduration=2.940754869 podStartE2EDuration="22.813983919s" totalImagesPullingTime="19.87322905s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:45.813979152 +0000 UTC m=+22.659843857" watchObservedRunningTime="2026-07-06 20:14:45.813983919 +0000 UTC m=+22.659848624" Jul 06 20:14:45.834066 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:45.834019 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" podStartSLOduration=21.834001918 podStartE2EDuration="21.834001918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:14:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:45.83373898 +0000 UTC m=+22.679603685" watchObservedRunningTime="2026-07-06 20:14:45.834001918 +0000 UTC m=+22.679866625" Jul 06 20:14:46.759108 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:46.759075 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q2nlz" event={"ID":"1a44f5a2-9953-4ae9-b0be-769c3f4e0d96","Type":"ContainerStarted","Data":"339079a9f7a274c58d1d25eb68d1a678c161bae86dae6f3526fafeaf9ef01f81"} Jul 06 20:14:46.760515 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:46.760494 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tlhdp" event={"ID":"1c6ffb1c-9153-4f2d-83ba-967ed6a026a1","Type":"ContainerStarted","Data":"5721bfc36ba23956d12db706673632a9ef76efe65361983ae42e6ccb31950ea7"} Jul 06 20:14:46.761923 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:46.761900 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tsptr" event={"ID":"1dbb2575-74b1-4231-bfd2-116630ca8579","Type":"ContainerStarted","Data":"744444cba100b251b0de3db3a9fab15e6a6cd5576a5fab93d6984f1ede5808b8"} Jul 06 20:14:46.763394 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:46.763374 2566 generic.go:356] "Generic (PLEG): container finished" podID="12b0e04f-1230-4820-a7ce-93a5ce83ef68" containerID="06edd45e9b94336a953ba3c0c0daf264ab5deb3f7a54f43e2d14644b2148c280" exitCode=0 Jul 06 20:14:46.763479 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:46.763439 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb74l" event={"ID":"12b0e04f-1230-4820-a7ce-93a5ce83ef68","Type":"ContainerDied","Data":"06edd45e9b94336a953ba3c0c0daf264ab5deb3f7a54f43e2d14644b2148c280"} Jul 06 20:14:46.764999 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:46.764978 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" event={"ID":"8896ec65-47bb-44a1-8790-6a3883dbf0f5","Type":"ContainerStarted","Data":"409210253be736b58a62b1c3854aac9016786495aaa7baff812b05debd5fe155"} Jul 06 20:14:46.768149 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:46.768126 2566 generic.go:356] "Generic (PLEG): container finished" podID="8c577c025dd62eaa24f491393f35ac71" containerID="d8daf712d849192cc650241f2c89f323112746210f7ca94200b83c1b89c3917e" exitCode=0 Jul 06 20:14:46.768238 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:46.768218 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" event={"ID":"8c577c025dd62eaa24f491393f35ac71","Type":"ContainerDied","Data":"d8daf712d849192cc650241f2c89f323112746210f7ca94200b83c1b89c3917e"} Jul 06 20:14:46.775981 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:46.775943 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-q2nlz" podStartSLOduration=4.526906223 podStartE2EDuration="23.775930026s" totalImagesPullingTime="19.249023803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:46.775906536 +0000 UTC m=+23.621771242" watchObservedRunningTime="2026-07-06 20:14:46.775930026 +0000 UTC m=+23.621794714" Jul 06 20:14:46.813614 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:46.813570 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tsptr" podStartSLOduration=3.258653367 podStartE2EDuration="22.813535815s" totalImagesPullingTime="19.554882448s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:46.792733534 +0000 UTC m=+23.638598241" watchObservedRunningTime="2026-07-06 20:14:46.813535815 +0000 UTC m=+23.659400526" Jul 06 20:14:46.814213 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:46.814172 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tlhdp" podStartSLOduration=3.068219236 podStartE2EDuration="22.814161919s" totalImagesPullingTime="19.745942683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:46.813904716 +0000 UTC m=+23.659769422" watchObservedRunningTime="2026-07-06 20:14:46.814161919 +0000 UTC m=+23.660026623" Jul 06 20:14:47.525235 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.525084 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:47.525696 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.525678 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:47.560230 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.560209 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 20:14:47.663287 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.663234 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:47.663375 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.663240 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:47.663375 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:47.663329 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:47.663535 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.663239 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:47.663535 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:47.663399 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:47.663535 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:47.663508 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:47.773201 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.773167 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" event={"ID":"8896ec65-47bb-44a1-8790-6a3883dbf0f5","Type":"ContainerStarted","Data":"b54b401bd7ca99216217b6cda526d99d62ce5c87653ecd1cc0958713bce1c16a"} Jul 06 20:14:47.775260 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.775235 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" event={"ID":"8c577c025dd62eaa24f491393f35ac71","Type":"ContainerStarted","Data":"3e2bc484bb2fc478a7bdb1d6fb208db1310e2aa0263096434a93606a2692b8ca"} Jul 06 20:14:47.776072 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.776054 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:47.776470 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.776452 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q2nlz" Jul 06 20:14:47.779264 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.779175 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T20:14:47.560226216Z","UUID":"bc191b58-b7a6-4ef6-a2ab-90c2c9c29b25","Handler":null,"Name":"","Endpoint":""} Jul 06 20:14:47.780956 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.780937 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 20:14:47.781041 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.780970 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 20:14:47.792955 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:47.792912 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" podStartSLOduration=23.792899470000002 podStartE2EDuration="23.79289947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:14:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:47.792308134 +0000 UTC m=+24.638172840" watchObservedRunningTime="2026-07-06 20:14:47.79289947 +0000 UTC m=+24.638764175" Jul 06 20:14:48.780602 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:48.780564 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" event={"ID":"ec562f06-afde-4298-bd3b-4b00bd4826b5","Type":"ContainerStarted","Data":"743e77db4f5b625275eb3997e6b883f90cc778a75697aeed485756e9f24368e0"} Jul 06 20:14:49.663118 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:49.663089 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:49.663118 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:49.663109 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:49.663384 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:49.663204 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:49.663384 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:49.663275 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:49.663384 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:49.663337 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:49.663534 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:49.663447 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:49.784476 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:49.784441 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" event={"ID":"8896ec65-47bb-44a1-8790-6a3883dbf0f5","Type":"ContainerStarted","Data":"78568b462f6b494d2d058fc8cc2cc6424c0d09aea3d59e5e4804f17507c6a710"} Jul 06 20:14:49.812648 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:49.812608 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-q2kbg" podStartSLOduration=3.981564725 podStartE2EDuration="26.812595267s" totalImagesPullingTime="22.831030542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:14:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:49.81245366 +0000 UTC m=+26.658318380" watchObservedRunningTime="2026-07-06 20:14:49.812595267 +0000 UTC m=+26.658459972" Jul 06 20:14:50.790864 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:50.790649 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" event={"ID":"ec562f06-afde-4298-bd3b-4b00bd4826b5","Type":"ContainerStarted","Data":"22c98f701b6da72f1ac414c246f5193ca35d7dfa3563996916477f48f8b572dd"} Jul 06 20:14:50.824848 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:50.824791 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" podStartSLOduration=6.724657125 podStartE2EDuration="26.8247725s" totalImagesPullingTime="20.100115375s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:50.824414071 +0000 UTC m=+27.670278790" watchObservedRunningTime="2026-07-06 20:14:50.8247725 +0000 UTC m=+27.670637206" Jul 06 20:14:51.663151 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:51.663122 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:51.663151 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:51.663140 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:51.663376 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:51.663120 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:51.663376 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:51.663271 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:51.663376 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:51.663355 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:51.663519 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:51.663452 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:51.793054 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:51.793024 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:51.793733 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:51.793145 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:51.793733 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:51.793363 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:51.810865 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:51.810828 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:51.811010 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:51.810983 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:14:52.239288 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:52.239251 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5slw5"] Jul 06 20:14:52.239436 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:52.239420 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:52.239573 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:52.239534 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:52.242112 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:52.242073 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tjmw4"] Jul 06 20:14:52.242225 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:52.242175 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:52.242265 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:52.242250 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:52.246212 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:52.246185 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r4xhr"] Jul 06 20:14:52.246335 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:52.246311 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:52.246453 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:52.246432 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:53.663691 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:53.663638 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:53.664094 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:53.663725 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:53.664094 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:53.663762 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:53.664094 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:53.663799 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:53.664094 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:53.663822 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:53.664094 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:53.663865 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:53.797405 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:53.797377 2566 generic.go:356] "Generic (PLEG): container finished" podID="12b0e04f-1230-4820-a7ce-93a5ce83ef68" containerID="16dd513770c7b75a265be16b0e0a3b376822437ecb09961f8a9839a18a5462b7" exitCode=0 Jul 06 20:14:53.797566 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:53.797454 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb74l" event={"ID":"12b0e04f-1230-4820-a7ce-93a5ce83ef68","Type":"ContainerDied","Data":"16dd513770c7b75a265be16b0e0a3b376822437ecb09961f8a9839a18a5462b7"} Jul 06 20:14:55.665066 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:55.664881 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:55.665461 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:55.664942 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:55.665461 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:55.665130 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:55.665461 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:55.664965 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:55.665461 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:55.665201 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:55.665461 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:55.665328 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:55.803929 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:55.803893 2566 generic.go:356] "Generic (PLEG): container finished" podID="12b0e04f-1230-4820-a7ce-93a5ce83ef68" containerID="1d9d21d3a4e957e82b6cd41c06f55743b311f71f3ecf8ee535054f2ffda80263" exitCode=0 Jul 06 20:14:55.804068 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:55.803950 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb74l" event={"ID":"12b0e04f-1230-4820-a7ce-93a5ce83ef68","Type":"ContainerDied","Data":"1d9d21d3a4e957e82b6cd41c06f55743b311f71f3ecf8ee535054f2ffda80263"} Jul 06 20:14:56.369347 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:56.369305 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:56.369509 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:56.369427 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:56.369509 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:56.369474 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs podName:e865c7c9-be01-4556-9120-f629f6a64fc0 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:28.369462233 +0000 UTC m=+65.215326929 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs") pod "network-metrics-daemon-r4xhr" (UID: "e865c7c9-be01-4556-9120-f629f6a64fc0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:14:56.470203 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:56.470165 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:56.470392 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:56.470220 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvd9\" (UniqueName: \"kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9\") pod \"network-check-target-5slw5\" (UID: \"812aa8ab-4982-4bba-9655-a5d69680cc8b\") " pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:56.470392 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:56.470320 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:56.470392 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:56.470338 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:14:56.470392 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:56.470355 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:14:56.470392 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:56.470367 2566 projected.go:196] Error preparing data for projected volume kube-api-access-twvd9 for pod openshift-network-diagnostics/network-check-target-5slw5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:56.470392 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:56.470394 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret podName:985b8c24-44e1-435e-96c0-e99a601b96c1 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:28.470375254 +0000 UTC m=+65.316239958 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret") pod "global-pull-secret-syncer-tjmw4" (UID: "985b8c24-44e1-435e-96c0-e99a601b96c1") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:14:56.470611 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:56.470411 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9 podName:812aa8ab-4982-4bba-9655-a5d69680cc8b nodeName:}" failed. No retries permitted until 2026-07-06 20:15:28.470403485 +0000 UTC m=+65.316268171 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-twvd9" (UniqueName: "kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9") pod "network-check-target-5slw5" (UID: "812aa8ab-4982-4bba-9655-a5d69680cc8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:14:57.665658 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:57.665632 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:57.666037 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:57.665634 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:57.666037 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:57.665746 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tjmw4" podUID="985b8c24-44e1-435e-96c0-e99a601b96c1" Jul 06 20:14:57.666037 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:57.665634 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:57.666037 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:57.665807 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5slw5" podUID="812aa8ab-4982-4bba-9655-a5d69680cc8b" Jul 06 20:14:57.666037 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:57.665871 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4xhr" podUID="e865c7c9-be01-4556-9120-f629f6a64fc0" Jul 06 20:14:57.810155 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:57.810092 2566 generic.go:356] "Generic (PLEG): container finished" podID="12b0e04f-1230-4820-a7ce-93a5ce83ef68" containerID="3bf2d5e2949ebe408b726af2ad2b6d3acce5d57247a93e0cd8a42075ca099e6a" exitCode=0 Jul 06 20:14:57.810155 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:57.810138 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb74l" event={"ID":"12b0e04f-1230-4820-a7ce-93a5ce83ef68","Type":"ContainerDied","Data":"3bf2d5e2949ebe408b726af2ad2b6d3acce5d57247a93e0cd8a42075ca099e6a"} Jul 06 20:14:57.948713 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:57.948694 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeReady" Jul 06 20:14:57.948857 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:57.948802 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 20:14:57.990772 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:57.990753 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c4ddc6c69-6q7pm"] Jul 06 20:14:58.005600 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.005580 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z"] Jul 06 20:14:58.005725 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.005706 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.008670 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.008650 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 20:14:58.009345 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.009318 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 20:14:58.009423 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.009316 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 20:14:58.009423 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.009360 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xfxxg\"" Jul 06 20:14:58.017109 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.017051 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r6k8b"] Jul 06 20:14:58.017188 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.017135 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" Jul 06 20:14:58.019380 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.019363 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 20:14:58.021502 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.021485 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-4pcqv\"" Jul 06 20:14:58.021502 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.021493 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:14:58.021666 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.021485 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 20:14:58.021779 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.021764 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 20:14:58.021948 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.021927 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 20:14:58.030971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.030491 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76"] Jul 06 20:14:58.030971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.030646 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r6k8b" Jul 06 20:14:58.035501 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.035478 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:14:58.035848 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.035826 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tzmkh\"" Jul 06 20:14:58.036151 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.036133 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 20:14:58.048815 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.048788 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-g6t88"] Jul 06 20:14:58.049147 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.048911 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" Jul 06 20:14:58.051439 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.051415 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 06 20:14:58.052161 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.052142 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-npr7l\"" Jul 06 20:14:58.052813 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.052792 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:14:58.052952 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.052936 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 06 20:14:58.053040 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.052801 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 06 20:14:58.061106 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.061065 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz"] Jul 06 20:14:58.061183 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.061166 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.064479 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.064463 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-fw6gh\"" Jul 06 20:14:58.064706 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.064531 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 20:14:58.064706 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.064666 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 20:14:58.064706 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.064685 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:14:58.065215 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.065197 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 20:14:58.075502 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.075482 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 20:14:58.075621 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.075539 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r6k8b"] Jul 06 20:14:58.075621 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.075588 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z"] Jul 06 20:14:58.075621 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.075611 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76"] Jul 06 20:14:58.075779 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.075639 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-ck4lm"] Jul 06 20:14:58.075779 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.075671 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:14:58.078401 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.078383 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 20:14:58.078488 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.078431 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pmsg9\"" Jul 06 20:14:58.078845 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.078828 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:14:58.078938 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.078858 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 20:14:58.081767 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.081464 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.081767 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.081494 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-certificates\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.081767 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.081570 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-trusted-ca\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.081767 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.081617 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-bound-sa-token\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.081767 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.081648 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-image-registry-private-configuration\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.081767 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.081679 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-installation-pull-secrets\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.081767 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.081711 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2188220-020e-4d49-95c4-a14e2aba5d35-ca-trust-extracted\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.081767 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.081736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrx2s\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-kube-api-access-hrx2s\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.086177 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.086159 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-g6t88"] Jul 06 20:14:58.086243 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.086196 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq"] Jul 06 20:14:58.086278 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.086246 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-ck4lm" Jul 06 20:14:58.089920 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.089902 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qhz2r\"" Jul 06 20:14:58.090023 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.089951 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 20:14:58.090179 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.090159 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 20:14:58.098296 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.098279 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c4ddc6c69-6q7pm"] Jul 06 20:14:58.098374 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.098306 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz"] Jul 06 20:14:58.098374 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.098334 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ckhsf"] Jul 06 20:14:58.098439 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.098384 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:14:58.100907 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.100886 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 06 20:14:58.100978 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.100887 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7kdgf\"" Jul 06 20:14:58.101117 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.101105 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 06 20:14:58.112730 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.112712 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ckhsf" Jul 06 20:14:58.112805 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.112712 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq"] Jul 06 20:14:58.112805 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.112774 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-ck4lm"] Jul 06 20:14:58.112805 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.112791 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ckhsf"] Jul 06 20:14:58.112900 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.112838 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x27pv"] Jul 06 20:14:58.115333 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.115319 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 20:14:58.115441 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.115351 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 20:14:58.115441 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.115356 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 20:14:58.115592 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.115524 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9vhmq\"" Jul 06 20:14:58.118856 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.118840 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.121766 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.121748 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 20:14:58.122073 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.122048 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 20:14:58.122161 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.122113 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 20:14:58.122161 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.122150 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hk74q\"" Jul 06 20:14:58.122313 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.122292 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 20:14:58.127817 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.127798 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x27pv"] Jul 06 20:14:58.182183 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182155 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85dr6\" (UniqueName: \"kubernetes.io/projected/5dd21d42-e317-4776-9822-2dbb049973f2-kube-api-access-85dr6\") pod \"kube-storage-version-migrator-operator-846bf668f7-w6b76\" (UID: \"5dd21d42-e317-4776-9822-2dbb049973f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" Jul 06 20:14:58.182345 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mgrq\" (UniqueName: \"kubernetes.io/projected/c582958d-acac-453d-8a5b-eb998905e4b0-kube-api-access-4mgrq\") pod \"volume-data-source-validator-5d456fdbcd-r6k8b\" (UID: \"c582958d-acac-453d-8a5b-eb998905e4b0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r6k8b" Jul 06 20:14:58.182345 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182222 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.182345 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-certificates\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.182345 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4vc2\" (UniqueName: \"kubernetes.io/projected/40e2374e-aadf-48c9-8fe8-808402f09282-kube-api-access-g4vc2\") pod \"cluster-samples-operator-56fcf86bf-4l4dz\" (UID: \"40e2374e-aadf-48c9-8fe8-808402f09282\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:14:58.182489 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182389 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-trusted-ca\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.182489 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.182421 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:14:58.182489 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.182435 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c4ddc6c69-6q7pm: secret "image-registry-tls" not found Jul 06 20:14:58.182489 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182435 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62d6ada4-8960-4e60-a605-9621138b9c8d-serving-cert\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.182489 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.182484 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls podName:a2188220-020e-4d49-95c4-a14e2aba5d35 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:58.682466268 +0000 UTC m=+35.528331007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls") pod "image-registry-7c4ddc6c69-6q7pm" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35") : secret "image-registry-tls" not found Jul 06 20:14:58.182768 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62d6ada4-8960-4e60-a605-9621138b9c8d-config\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.182768 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182574 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-bound-sa-token\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.182768 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182599 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5dd21d42-e317-4776-9822-2dbb049973f2-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-w6b76\" (UID: \"5dd21d42-e317-4776-9822-2dbb049973f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" Jul 06 20:14:58.182768 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182623 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-9pq4z\" (UID: \"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" Jul 06 20:14:58.182768 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182645 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89-config\") pod \"service-ca-operator-5d46d9ffd4-9pq4z\" (UID: \"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" Jul 06 20:14:58.182768 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182673 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v466v\" (UniqueName: \"kubernetes.io/projected/5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89-kube-api-access-v466v\") pod \"service-ca-operator-5d46d9ffd4-9pq4z\" (UID: \"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" Jul 06 20:14:58.182768 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182701 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-image-registry-private-configuration\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.182768 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182726 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:14:58.182768 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182751 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98k6f\" (UniqueName: \"kubernetes.io/projected/4da47a5d-5846-4992-8962-66010b22bd3c-kube-api-access-98k6f\") pod \"network-check-source-989f8b8dc-ck4lm\" (UID: \"4da47a5d-5846-4992-8962-66010b22bd3c\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-ck4lm" Jul 06 20:14:58.183154 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-installation-pull-secrets\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.183154 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182830 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dd21d42-e317-4776-9822-2dbb049973f2-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-w6b76\" (UID: \"5dd21d42-e317-4776-9822-2dbb049973f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" Jul 06 20:14:58.183154 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182866 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52e1eff4-f7e7-4307-a72b-2bd3b2e58241-cert\") pod \"ingress-canary-ckhsf\" (UID: \"52e1eff4-f7e7-4307-a72b-2bd3b2e58241\") " pod="openshift-ingress-canary/ingress-canary-ckhsf" Jul 06 20:14:58.183154 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182903 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2188220-020e-4d49-95c4-a14e2aba5d35-ca-trust-extracted\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.183154 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182907 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-certificates\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.183154 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182928 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrx2s\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-kube-api-access-hrx2s\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.183154 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182964 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62d6ada4-8960-4e60-a605-9621138b9c8d-trusted-ca\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.183154 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.182998 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr4mg\" (UniqueName: \"kubernetes.io/projected/62d6ada4-8960-4e60-a605-9621138b9c8d-kube-api-access-zr4mg\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.183154 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.183023 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/81143b00-2314-4ecf-ac5e-fd92e429639c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:14:58.183154 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.183083 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsrpl\" (UniqueName: \"kubernetes.io/projected/52e1eff4-f7e7-4307-a72b-2bd3b2e58241-kube-api-access-nsrpl\") pod \"ingress-canary-ckhsf\" (UID: \"52e1eff4-f7e7-4307-a72b-2bd3b2e58241\") " pod="openshift-ingress-canary/ingress-canary-ckhsf" Jul 06 20:14:58.183476 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.183175 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2188220-020e-4d49-95c4-a14e2aba5d35-ca-trust-extracted\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.183476 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.183187 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l4dz\" (UID: \"40e2374e-aadf-48c9-8fe8-808402f09282\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:14:58.183476 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.183467 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-trusted-ca\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.186894 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.186870 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-image-registry-private-configuration\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.186984 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.186873 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-installation-pull-secrets\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.197799 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.197769 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-bound-sa-token\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.198045 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.198026 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrx2s\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-kube-api-access-hrx2s\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.284211 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-config-volume\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.284305 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284229 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5dd21d42-e317-4776-9822-2dbb049973f2-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-w6b76\" (UID: \"5dd21d42-e317-4776-9822-2dbb049973f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" Jul 06 20:14:58.284305 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62d6ada4-8960-4e60-a605-9621138b9c8d-config\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.284305 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284282 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52e1eff4-f7e7-4307-a72b-2bd3b2e58241-cert\") pod \"ingress-canary-ckhsf\" (UID: \"52e1eff4-f7e7-4307-a72b-2bd3b2e58241\") " pod="openshift-ingress-canary/ingress-canary-ckhsf" Jul 06 20:14:58.284466 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284320 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98k6f\" (UniqueName: \"kubernetes.io/projected/4da47a5d-5846-4992-8962-66010b22bd3c-kube-api-access-98k6f\") pod \"network-check-source-989f8b8dc-ck4lm\" (UID: \"4da47a5d-5846-4992-8962-66010b22bd3c\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-ck4lm" Jul 06 20:14:58.284466 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284358 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr4mg\" (UniqueName: \"kubernetes.io/projected/62d6ada4-8960-4e60-a605-9621138b9c8d-kube-api-access-zr4mg\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.284535 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284497 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsrpl\" (UniqueName: \"kubernetes.io/projected/52e1eff4-f7e7-4307-a72b-2bd3b2e58241-kube-api-access-nsrpl\") pod \"ingress-canary-ckhsf\" (UID: \"52e1eff4-f7e7-4307-a72b-2bd3b2e58241\") " pod="openshift-ingress-canary/ingress-canary-ckhsf" Jul 06 20:14:58.284614 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284536 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62d6ada4-8960-4e60-a605-9621138b9c8d-trusted-ca\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.284614 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284579 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.284614 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284608 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-tmp-dir\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.284813 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284680 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l4dz\" (UID: \"40e2374e-aadf-48c9-8fe8-808402f09282\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:14:58.284813 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284714 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mgrq\" (UniqueName: \"kubernetes.io/projected/c582958d-acac-453d-8a5b-eb998905e4b0-kube-api-access-4mgrq\") pod \"volume-data-source-validator-5d456fdbcd-r6k8b\" (UID: \"c582958d-acac-453d-8a5b-eb998905e4b0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r6k8b" Jul 06 20:14:58.284813 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284757 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5dd21d42-e317-4776-9822-2dbb049973f2-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-w6b76\" (UID: \"5dd21d42-e317-4776-9822-2dbb049973f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" Jul 06 20:14:58.284813 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284765 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4vc2\" (UniqueName: \"kubernetes.io/projected/40e2374e-aadf-48c9-8fe8-808402f09282-kube-api-access-g4vc2\") pod \"cluster-samples-operator-56fcf86bf-4l4dz\" (UID: \"40e2374e-aadf-48c9-8fe8-808402f09282\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:14:58.284813 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284791 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhwtd\" (UniqueName: \"kubernetes.io/projected/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-kube-api-access-lhwtd\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.285108 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.284845 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:14:58.285108 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62d6ada4-8960-4e60-a605-9621138b9c8d-serving-cert\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.285108 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284909 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dd21d42-e317-4776-9822-2dbb049973f2-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-w6b76\" (UID: \"5dd21d42-e317-4776-9822-2dbb049973f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" Jul 06 20:14:58.285108 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.284943 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls podName:40e2374e-aadf-48c9-8fe8-808402f09282 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:58.784924999 +0000 UTC m=+35.630789695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-4l4dz" (UID: "40e2374e-aadf-48c9-8fe8-808402f09282") : secret "samples-operator-tls" not found Jul 06 20:14:58.285108 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.284945 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62d6ada4-8960-4e60-a605-9621138b9c8d-config\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.285108 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.285063 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89-config\") pod \"service-ca-operator-5d46d9ffd4-9pq4z\" (UID: \"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" Jul 06 20:14:58.285429 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.285186 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:14:58.285429 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.285223 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-9pq4z\" (UID: \"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" Jul 06 20:14:58.285429 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.285248 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v466v\" (UniqueName: \"kubernetes.io/projected/5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89-kube-api-access-v466v\") pod \"service-ca-operator-5d46d9ffd4-9pq4z\" (UID: \"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" Jul 06 20:14:58.285429 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.285278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/81143b00-2314-4ecf-ac5e-fd92e429639c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:14:58.285429 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.285316 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85dr6\" (UniqueName: \"kubernetes.io/projected/5dd21d42-e317-4776-9822-2dbb049973f2-kube-api-access-85dr6\") pod \"kube-storage-version-migrator-operator-846bf668f7-w6b76\" (UID: \"5dd21d42-e317-4776-9822-2dbb049973f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" Jul 06 20:14:58.285429 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.285347 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62d6ada4-8960-4e60-a605-9621138b9c8d-trusted-ca\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.285429 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.285353 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:14:58.285789 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.285447 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert podName:81143b00-2314-4ecf-ac5e-fd92e429639c nodeName:}" failed. No retries permitted until 2026-07-06 20:14:58.785428562 +0000 UTC m=+35.631293268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zqtxq" (UID: "81143b00-2314-4ecf-ac5e-fd92e429639c") : secret "networking-console-plugin-cert" not found Jul 06 20:14:58.285789 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.285733 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89-config\") pod \"service-ca-operator-5d46d9ffd4-9pq4z\" (UID: \"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" Jul 06 20:14:58.286077 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.286055 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/81143b00-2314-4ecf-ac5e-fd92e429639c-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:14:58.287016 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.286989 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52e1eff4-f7e7-4307-a72b-2bd3b2e58241-cert\") pod \"ingress-canary-ckhsf\" (UID: \"52e1eff4-f7e7-4307-a72b-2bd3b2e58241\") " pod="openshift-ingress-canary/ingress-canary-ckhsf" Jul 06 20:14:58.287117 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.287097 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dd21d42-e317-4776-9822-2dbb049973f2-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-w6b76\" (UID: \"5dd21d42-e317-4776-9822-2dbb049973f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" Jul 06 20:14:58.287422 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.287400 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62d6ada4-8960-4e60-a605-9621138b9c8d-serving-cert\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.287535 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.287521 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-9pq4z\" (UID: \"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" Jul 06 20:14:58.294095 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.294064 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsrpl\" (UniqueName: \"kubernetes.io/projected/52e1eff4-f7e7-4307-a72b-2bd3b2e58241-kube-api-access-nsrpl\") pod \"ingress-canary-ckhsf\" (UID: \"52e1eff4-f7e7-4307-a72b-2bd3b2e58241\") " pod="openshift-ingress-canary/ingress-canary-ckhsf" Jul 06 20:14:58.294353 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.294334 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mgrq\" (UniqueName: \"kubernetes.io/projected/c582958d-acac-453d-8a5b-eb998905e4b0-kube-api-access-4mgrq\") pod \"volume-data-source-validator-5d456fdbcd-r6k8b\" (UID: \"c582958d-acac-453d-8a5b-eb998905e4b0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r6k8b" Jul 06 20:14:58.294353 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.294340 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98k6f\" (UniqueName: \"kubernetes.io/projected/4da47a5d-5846-4992-8962-66010b22bd3c-kube-api-access-98k6f\") pod \"network-check-source-989f8b8dc-ck4lm\" (UID: \"4da47a5d-5846-4992-8962-66010b22bd3c\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-ck4lm" Jul 06 20:14:58.294490 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.294469 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr4mg\" (UniqueName: \"kubernetes.io/projected/62d6ada4-8960-4e60-a605-9621138b9c8d-kube-api-access-zr4mg\") pod \"console-operator-68b757fb67-g6t88\" (UID: \"62d6ada4-8960-4e60-a605-9621138b9c8d\") " pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.295243 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.295220 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85dr6\" (UniqueName: \"kubernetes.io/projected/5dd21d42-e317-4776-9822-2dbb049973f2-kube-api-access-85dr6\") pod \"kube-storage-version-migrator-operator-846bf668f7-w6b76\" (UID: \"5dd21d42-e317-4776-9822-2dbb049973f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" Jul 06 20:14:58.295580 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.295536 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4vc2\" (UniqueName: \"kubernetes.io/projected/40e2374e-aadf-48c9-8fe8-808402f09282-kube-api-access-g4vc2\") pod \"cluster-samples-operator-56fcf86bf-4l4dz\" (UID: \"40e2374e-aadf-48c9-8fe8-808402f09282\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:14:58.295804 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.295789 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v466v\" (UniqueName: \"kubernetes.io/projected/5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89-kube-api-access-v466v\") pod \"service-ca-operator-5d46d9ffd4-9pq4z\" (UID: \"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" Jul 06 20:14:58.326368 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.326328 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" Jul 06 20:14:58.341458 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.341442 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r6k8b" Jul 06 20:14:58.357862 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.357739 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" Jul 06 20:14:58.369769 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.369701 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:14:58.385335 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.385309 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-6q7sk"] Jul 06 20:14:58.386035 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.386016 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-tmp-dir\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.386128 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.386092 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhwtd\" (UniqueName: \"kubernetes.io/projected/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-kube-api-access-lhwtd\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.386215 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.386190 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-config-volume\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.386300 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.386258 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.386391 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.386378 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:14:58.386465 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.386418 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-tmp-dir\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.386538 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.386481 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls podName:e23f0332-a4f0-4638-8ac3-a9d89fa3564a nodeName:}" failed. No retries permitted until 2026-07-06 20:14:58.886453423 +0000 UTC m=+35.732318109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls") pod "dns-default-x27pv" (UID: "e23f0332-a4f0-4638-8ac3-a9d89fa3564a") : secret "dns-default-metrics-tls" not found Jul 06 20:14:58.387009 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.386891 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-config-volume\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.395032 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.395010 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-ck4lm" Jul 06 20:14:58.406384 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.406319 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhwtd\" (UniqueName: \"kubernetes.io/projected/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-kube-api-access-lhwtd\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.409789 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.409001 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6q7sk" Jul 06 20:14:58.411800 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.411749 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rq42z\"" Jul 06 20:14:58.420656 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.420607 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ckhsf" Jul 06 20:14:58.489397 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.488940 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5d7c76e4-8c48-47ca-9514-6d26ac324460-hosts-file\") pod \"node-resolver-6q7sk\" (UID: \"5d7c76e4-8c48-47ca-9514-6d26ac324460\") " pod="openshift-dns/node-resolver-6q7sk" Jul 06 20:14:58.489397 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.488992 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5tl4\" (UniqueName: \"kubernetes.io/projected/5d7c76e4-8c48-47ca-9514-6d26ac324460-kube-api-access-r5tl4\") pod \"node-resolver-6q7sk\" (UID: \"5d7c76e4-8c48-47ca-9514-6d26ac324460\") " pod="openshift-dns/node-resolver-6q7sk" Jul 06 20:14:58.489397 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.489047 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d7c76e4-8c48-47ca-9514-6d26ac324460-tmp-dir\") pod \"node-resolver-6q7sk\" (UID: \"5d7c76e4-8c48-47ca-9514-6d26ac324460\") " pod="openshift-dns/node-resolver-6q7sk" Jul 06 20:14:58.571022 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.570902 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z"] Jul 06 20:14:58.583041 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.582966 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r6k8b"] Jul 06 20:14:58.592037 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.590026 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5tl4\" (UniqueName: \"kubernetes.io/projected/5d7c76e4-8c48-47ca-9514-6d26ac324460-kube-api-access-r5tl4\") pod \"node-resolver-6q7sk\" (UID: \"5d7c76e4-8c48-47ca-9514-6d26ac324460\") " pod="openshift-dns/node-resolver-6q7sk" Jul 06 20:14:58.592037 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.590113 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d7c76e4-8c48-47ca-9514-6d26ac324460-tmp-dir\") pod \"node-resolver-6q7sk\" (UID: \"5d7c76e4-8c48-47ca-9514-6d26ac324460\") " pod="openshift-dns/node-resolver-6q7sk" Jul 06 20:14:58.592037 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.590260 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5d7c76e4-8c48-47ca-9514-6d26ac324460-hosts-file\") pod \"node-resolver-6q7sk\" (UID: \"5d7c76e4-8c48-47ca-9514-6d26ac324460\") " pod="openshift-dns/node-resolver-6q7sk" Jul 06 20:14:58.592037 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.590386 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5d7c76e4-8c48-47ca-9514-6d26ac324460-hosts-file\") pod \"node-resolver-6q7sk\" (UID: \"5d7c76e4-8c48-47ca-9514-6d26ac324460\") " pod="openshift-dns/node-resolver-6q7sk" Jul 06 20:14:58.592037 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.591237 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d7c76e4-8c48-47ca-9514-6d26ac324460-tmp-dir\") pod \"node-resolver-6q7sk\" (UID: \"5d7c76e4-8c48-47ca-9514-6d26ac324460\") " pod="openshift-dns/node-resolver-6q7sk" Jul 06 20:14:58.598228 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:58.597650 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc582958d_acac_453d_8a5b_eb998905e4b0.slice/crio-a51951cfd0b29d795799bad9622fc73fc3479fa27a3e3543a1c096f7eaec223e WatchSource:0}: Error finding container a51951cfd0b29d795799bad9622fc73fc3479fa27a3e3543a1c096f7eaec223e: Status 404 returned error can't find the container with id a51951cfd0b29d795799bad9622fc73fc3479fa27a3e3543a1c096f7eaec223e Jul 06 20:14:58.600820 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.600432 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5tl4\" (UniqueName: \"kubernetes.io/projected/5d7c76e4-8c48-47ca-9514-6d26ac324460-kube-api-access-r5tl4\") pod \"node-resolver-6q7sk\" (UID: \"5d7c76e4-8c48-47ca-9514-6d26ac324460\") " pod="openshift-dns/node-resolver-6q7sk" Jul 06 20:14:58.653530 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.653498 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76"] Jul 06 20:14:58.657030 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:58.656995 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd21d42_e317_4776_9822_2dbb049973f2.slice/crio-6b288e3c3a6b04f0cb3191082d234ec98c622c61e5462f754b8dde581010765b WatchSource:0}: Error finding container 6b288e3c3a6b04f0cb3191082d234ec98c622c61e5462f754b8dde581010765b: Status 404 returned error can't find the container with id 6b288e3c3a6b04f0cb3191082d234ec98c622c61e5462f754b8dde581010765b Jul 06 20:14:58.664421 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.664359 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ckhsf"] Jul 06 20:14:58.668461 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:58.668437 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e1eff4_f7e7_4307_a72b_2bd3b2e58241.slice/crio-e87e413435bc7e7949bf2a970c160d982c5dede9ef2e701dfb5cccc5af02b927 WatchSource:0}: Error finding container e87e413435bc7e7949bf2a970c160d982c5dede9ef2e701dfb5cccc5af02b927: Status 404 returned error can't find the container with id e87e413435bc7e7949bf2a970c160d982c5dede9ef2e701dfb5cccc5af02b927 Jul 06 20:14:58.675002 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.674963 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-g6t88"] Jul 06 20:14:58.679301 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:58.679278 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d6ada4_8960_4e60_a605_9621138b9c8d.slice/crio-a72022360d0a69631471508c19ccfa041f4a04176d6788e8a439112f15124f01 WatchSource:0}: Error finding container a72022360d0a69631471508c19ccfa041f4a04176d6788e8a439112f15124f01: Status 404 returned error can't find the container with id a72022360d0a69631471508c19ccfa041f4a04176d6788e8a439112f15124f01 Jul 06 20:14:58.682193 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.682162 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-ck4lm"] Jul 06 20:14:58.684725 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:58.684702 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4da47a5d_5846_4992_8962_66010b22bd3c.slice/crio-d7f27f52e0a7b56524a75f4519b33d612fa49ad8ec22034a82ddf1423f336673 WatchSource:0}: Error finding container d7f27f52e0a7b56524a75f4519b33d612fa49ad8ec22034a82ddf1423f336673: Status 404 returned error can't find the container with id d7f27f52e0a7b56524a75f4519b33d612fa49ad8ec22034a82ddf1423f336673 Jul 06 20:14:58.691728 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.691704 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:58.691869 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.691856 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:14:58.691923 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.691871 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c4ddc6c69-6q7pm: secret "image-registry-tls" not found Jul 06 20:14:58.691961 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.691922 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls podName:a2188220-020e-4d49-95c4-a14e2aba5d35 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:59.691901284 +0000 UTC m=+36.537765988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls") pod "image-registry-7c4ddc6c69-6q7pm" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35") : secret "image-registry-tls" not found Jul 06 20:14:58.720874 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.720856 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6q7sk" Jul 06 20:14:58.765251 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:14:58.765224 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7c76e4_8c48_47ca_9514_6d26ac324460.slice/crio-ac2775372bf1229a3a992a5cda974439438fb9d7546d0b291892bd4dcb0aa607 WatchSource:0}: Error finding container ac2775372bf1229a3a992a5cda974439438fb9d7546d0b291892bd4dcb0aa607: Status 404 returned error can't find the container with id ac2775372bf1229a3a992a5cda974439438fb9d7546d0b291892bd4dcb0aa607 Jul 06 20:14:58.792167 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.792136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l4dz\" (UID: \"40e2374e-aadf-48c9-8fe8-808402f09282\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:14:58.792291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.792225 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:14:58.792356 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.792291 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:14:58.792356 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.792339 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:14:58.792450 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.792361 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls podName:40e2374e-aadf-48c9-8fe8-808402f09282 nodeName:}" failed. No retries permitted until 2026-07-06 20:14:59.792341869 +0000 UTC m=+36.638206612 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-4l4dz" (UID: "40e2374e-aadf-48c9-8fe8-808402f09282") : secret "samples-operator-tls" not found Jul 06 20:14:58.792450 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.792391 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert podName:81143b00-2314-4ecf-ac5e-fd92e429639c nodeName:}" failed. No retries permitted until 2026-07-06 20:14:59.792375317 +0000 UTC m=+36.638240017 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zqtxq" (UID: "81143b00-2314-4ecf-ac5e-fd92e429639c") : secret "networking-console-plugin-cert" not found Jul 06 20:14:58.813373 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.813175 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" event={"ID":"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89","Type":"ContainerStarted","Data":"49523ec2663aec0aff954dc6ba7d0cab9bf6d4e88f4970dbfa802094665cb0a4"} Jul 06 20:14:58.814253 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.814223 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6q7sk" event={"ID":"5d7c76e4-8c48-47ca-9514-6d26ac324460","Type":"ContainerStarted","Data":"ac2775372bf1229a3a992a5cda974439438fb9d7546d0b291892bd4dcb0aa607"} Jul 06 20:14:58.815600 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.815401 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" event={"ID":"62d6ada4-8960-4e60-a605-9621138b9c8d","Type":"ContainerStarted","Data":"a72022360d0a69631471508c19ccfa041f4a04176d6788e8a439112f15124f01"} Jul 06 20:14:58.816670 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.816644 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-ck4lm" event={"ID":"4da47a5d-5846-4992-8962-66010b22bd3c","Type":"ContainerStarted","Data":"d7f27f52e0a7b56524a75f4519b33d612fa49ad8ec22034a82ddf1423f336673"} Jul 06 20:14:58.817776 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.817755 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ckhsf" event={"ID":"52e1eff4-f7e7-4307-a72b-2bd3b2e58241","Type":"ContainerStarted","Data":"e87e413435bc7e7949bf2a970c160d982c5dede9ef2e701dfb5cccc5af02b927"} Jul 06 20:14:58.819052 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.819023 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" event={"ID":"5dd21d42-e317-4776-9822-2dbb049973f2","Type":"ContainerStarted","Data":"6b288e3c3a6b04f0cb3191082d234ec98c622c61e5462f754b8dde581010765b"} Jul 06 20:14:58.820240 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.820221 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r6k8b" event={"ID":"c582958d-acac-453d-8a5b-eb998905e4b0","Type":"ContainerStarted","Data":"a51951cfd0b29d795799bad9622fc73fc3479fa27a3e3543a1c096f7eaec223e"} Jul 06 20:14:58.893584 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:58.893543 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:58.893722 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.893707 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:14:58.893814 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:58.893781 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls podName:e23f0332-a4f0-4638-8ac3-a9d89fa3564a nodeName:}" failed. No retries permitted until 2026-07-06 20:14:59.893761464 +0000 UTC m=+36.739626161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls") pod "dns-default-x27pv" (UID: "e23f0332-a4f0-4638-8ac3-a9d89fa3564a") : secret "dns-default-metrics-tls" not found Jul 06 20:14:59.672801 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.672769 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:14:59.673243 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.673035 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:14:59.673317 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.673306 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:14:59.676500 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.676470 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8g8p6\"" Jul 06 20:14:59.676760 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.676714 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 20:14:59.677097 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.677075 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 20:14:59.681259 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.680858 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ksdkk\"" Jul 06 20:14:59.700782 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.700762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:14:59.701094 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:59.701074 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:14:59.701178 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:59.701097 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c4ddc6c69-6q7pm: secret "image-registry-tls" not found Jul 06 20:14:59.701178 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:59.701160 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls podName:a2188220-020e-4d49-95c4-a14e2aba5d35 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:01.701141117 +0000 UTC m=+38.547005820 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls") pod "image-registry-7c4ddc6c69-6q7pm" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35") : secret "image-registry-tls" not found Jul 06 20:14:59.803588 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.802278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l4dz\" (UID: \"40e2374e-aadf-48c9-8fe8-808402f09282\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:14:59.803588 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.802498 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:14:59.803588 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:59.803269 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:14:59.803588 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:59.803328 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls podName:40e2374e-aadf-48c9-8fe8-808402f09282 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:01.803307318 +0000 UTC m=+38.649172015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-4l4dz" (UID: "40e2374e-aadf-48c9-8fe8-808402f09282") : secret "samples-operator-tls" not found Jul 06 20:14:59.803905 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:59.803648 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:14:59.803905 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:59.803698 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert podName:81143b00-2314-4ecf-ac5e-fd92e429639c nodeName:}" failed. No retries permitted until 2026-07-06 20:15:01.803682041 +0000 UTC m=+38.649546737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zqtxq" (UID: "81143b00-2314-4ecf-ac5e-fd92e429639c") : secret "networking-console-plugin-cert" not found Jul 06 20:14:59.829630 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.829017 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6q7sk" event={"ID":"5d7c76e4-8c48-47ca-9514-6d26ac324460","Type":"ContainerStarted","Data":"51426ba68d1d39d0a2b2307773b706c93e06513d75a84b2bc17059b675c7912d"} Jul 06 20:14:59.848987 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.848572 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6q7sk" podStartSLOduration=1.848539532 podStartE2EDuration="1.848539532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:14:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:59.846798394 +0000 UTC m=+36.692663121" watchObservedRunningTime="2026-07-06 20:14:59.848539532 +0000 UTC m=+36.694404234" Jul 06 20:14:59.904385 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:14:59.903722 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:14:59.904385 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:59.903977 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:14:59.904385 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:14:59.904039 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls podName:e23f0332-a4f0-4638-8ac3-a9d89fa3564a nodeName:}" failed. No retries permitted until 2026-07-06 20:15:01.904021159 +0000 UTC m=+38.749885856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls") pod "dns-default-x27pv" (UID: "e23f0332-a4f0-4638-8ac3-a9d89fa3564a") : secret "dns-default-metrics-tls" not found Jul 06 20:15:01.722888 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:01.722844 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:15:01.723324 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:01.723021 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:15:01.723324 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:01.723039 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c4ddc6c69-6q7pm: secret "image-registry-tls" not found Jul 06 20:15:01.723324 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:01.723102 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls podName:a2188220-020e-4d49-95c4-a14e2aba5d35 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:05.723082562 +0000 UTC m=+42.568947268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls") pod "image-registry-7c4ddc6c69-6q7pm" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35") : secret "image-registry-tls" not found Jul 06 20:15:01.824281 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:01.824248 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l4dz\" (UID: \"40e2374e-aadf-48c9-8fe8-808402f09282\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:15:01.824446 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:01.824316 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:15:01.824446 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:01.824411 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:15:01.824635 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:01.824481 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls podName:40e2374e-aadf-48c9-8fe8-808402f09282 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:05.824461021 +0000 UTC m=+42.670325782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-4l4dz" (UID: "40e2374e-aadf-48c9-8fe8-808402f09282") : secret "samples-operator-tls" not found Jul 06 20:15:01.824635 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:01.824413 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:15:01.824635 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:01.824601 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert podName:81143b00-2314-4ecf-ac5e-fd92e429639c nodeName:}" failed. No retries permitted until 2026-07-06 20:15:05.82458479 +0000 UTC m=+42.670449492 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zqtxq" (UID: "81143b00-2314-4ecf-ac5e-fd92e429639c") : secret "networking-console-plugin-cert" not found Jul 06 20:15:01.925837 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:01.925761 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:15:01.926041 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:01.925927 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:15:01.926041 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:01.925995 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls podName:e23f0332-a4f0-4638-8ac3-a9d89fa3564a nodeName:}" failed. No retries permitted until 2026-07-06 20:15:05.925978124 +0000 UTC m=+42.771842821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls") pod "dns-default-x27pv" (UID: "e23f0332-a4f0-4638-8ac3-a9d89fa3564a") : secret "dns-default-metrics-tls" not found Jul 06 20:15:05.755157 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:05.755121 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:15:05.755630 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:05.755274 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:15:05.755630 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:05.755293 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c4ddc6c69-6q7pm: secret "image-registry-tls" not found Jul 06 20:15:05.755630 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:05.755377 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls podName:a2188220-020e-4d49-95c4-a14e2aba5d35 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:13.755357581 +0000 UTC m=+50.601222282 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls") pod "image-registry-7c4ddc6c69-6q7pm" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35") : secret "image-registry-tls" not found Jul 06 20:15:05.856605 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:05.856515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l4dz\" (UID: \"40e2374e-aadf-48c9-8fe8-808402f09282\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:15:05.856752 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:05.856608 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:15:05.856752 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:05.856680 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:15:05.856752 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:05.856748 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls podName:40e2374e-aadf-48c9-8fe8-808402f09282 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:13.856728139 +0000 UTC m=+50.702592822 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-4l4dz" (UID: "40e2374e-aadf-48c9-8fe8-808402f09282") : secret "samples-operator-tls" not found Jul 06 20:15:05.856916 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:05.856751 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:15:05.856916 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:05.856812 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert podName:81143b00-2314-4ecf-ac5e-fd92e429639c nodeName:}" failed. No retries permitted until 2026-07-06 20:15:13.85679594 +0000 UTC m=+50.702660645 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zqtxq" (UID: "81143b00-2314-4ecf-ac5e-fd92e429639c") : secret "networking-console-plugin-cert" not found Jul 06 20:15:05.957328 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:05.957290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:15:05.957512 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:05.957448 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:15:05.957598 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:05.957514 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls podName:e23f0332-a4f0-4638-8ac3-a9d89fa3564a nodeName:}" failed. No retries permitted until 2026-07-06 20:15:13.957493979 +0000 UTC m=+50.803358661 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls") pod "dns-default-x27pv" (UID: "e23f0332-a4f0-4638-8ac3-a9d89fa3564a") : secret "dns-default-metrics-tls" not found Jul 06 20:15:07.856091 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.855172 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/0.log" Jul 06 20:15:07.856091 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.855214 2566 generic.go:356] "Generic (PLEG): container finished" podID="62d6ada4-8960-4e60-a605-9621138b9c8d" containerID="c3982048060959dc568510522d4d52df9e2865fc4d4911535daada38fea131dc" exitCode=255 Jul 06 20:15:07.856091 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.855294 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" event={"ID":"62d6ada4-8960-4e60-a605-9621138b9c8d","Type":"ContainerDied","Data":"c3982048060959dc568510522d4d52df9e2865fc4d4911535daada38fea131dc"} Jul 06 20:15:07.856091 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.855651 2566 scope.go:117] "RemoveContainer" containerID="c3982048060959dc568510522d4d52df9e2865fc4d4911535daada38fea131dc" Jul 06 20:15:07.861668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.861415 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-ck4lm" event={"ID":"4da47a5d-5846-4992-8962-66010b22bd3c","Type":"ContainerStarted","Data":"8badc90840aa7041806276197003352c1b07ea1a30a2914a5a865cd5ee209f8b"} Jul 06 20:15:07.864099 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.864035 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ckhsf" event={"ID":"52e1eff4-f7e7-4307-a72b-2bd3b2e58241","Type":"ContainerStarted","Data":"99b6c950441f26f6935605e8df596ebbfbd1855449bfb55499eeed48aedb77ab"} Jul 06 20:15:07.867006 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.866603 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" event={"ID":"5dd21d42-e317-4776-9822-2dbb049973f2","Type":"ContainerStarted","Data":"96f23355e96d850bfa5bb073fff459d39057db3a5274c094c68949c5c9271030"} Jul 06 20:15:07.872606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.872199 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r6k8b" event={"ID":"c582958d-acac-453d-8a5b-eb998905e4b0","Type":"ContainerStarted","Data":"415e5787353b6b8b242739b2f57e4b95b3608c7c1a1edd4b01d5c076c1103f11"} Jul 06 20:15:07.880511 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.880482 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" event={"ID":"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89","Type":"ContainerStarted","Data":"6ac15cd15ec78c13c8c09fbd3f48d594722984f0d66480313accb10e89b47c62"} Jul 06 20:15:07.904863 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.904233 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" podStartSLOduration=25.958996818 podStartE2EDuration="34.9042136s" totalImagesPullingTime="8.945216782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:07.903588574 +0000 UTC m=+44.749453279" watchObservedRunningTime="2026-07-06 20:15:07.9042136 +0000 UTC m=+44.750078305" Jul 06 20:15:07.927330 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.927284 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ckhsf" podStartSLOduration=1.9928582540000002 podStartE2EDuration="10.927251079s" totalImagesPullingTime="8.934392825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:07.926527644 +0000 UTC m=+44.772392372" watchObservedRunningTime="2026-07-06 20:15:07.927251079 +0000 UTC m=+44.773115785" Jul 06 20:15:07.969565 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.968902 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-r6k8b" podStartSLOduration=27.969993376 podStartE2EDuration="34.9688805s" totalImagesPullingTime="6.998887124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:07.947281883 +0000 UTC m=+44.793146592" watchObservedRunningTime="2026-07-06 20:15:07.9688805 +0000 UTC m=+44.814745207" Jul 06 20:15:07.992341 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.991821 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" podStartSLOduration=25.983719532 podStartE2EDuration="34.991798724s" totalImagesPullingTime="9.008079192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:07.990864446 +0000 UTC m=+44.836729150" watchObservedRunningTime="2026-07-06 20:15:07.991798724 +0000 UTC m=+44.837663433" Jul 06 20:15:07.992341 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:07.992165 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-ck4lm" podStartSLOduration=7.073576417 podStartE2EDuration="15.992153106s" totalImagesPullingTime="8.918576689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:07.969757382 +0000 UTC m=+44.815622083" watchObservedRunningTime="2026-07-06 20:15:07.992153106 +0000 UTC m=+44.838017811" Jul 06 20:15:08.370242 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.370202 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:15:08.370382 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.370280 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:15:08.885914 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.885887 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:15:08.886535 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.886517 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/0.log" Jul 06 20:15:08.886682 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.886579 2566 generic.go:356] "Generic (PLEG): container finished" podID="62d6ada4-8960-4e60-a605-9621138b9c8d" containerID="7a26af0ef24929c62a60edbf6a98b398c801ef6dcf6c3fba0ba4e7774de1ecf6" exitCode=255 Jul 06 20:15:08.886737 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.886677 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" event={"ID":"62d6ada4-8960-4e60-a605-9621138b9c8d","Type":"ContainerDied","Data":"7a26af0ef24929c62a60edbf6a98b398c801ef6dcf6c3fba0ba4e7774de1ecf6"} Jul 06 20:15:08.886791 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.886743 2566 scope.go:117] "RemoveContainer" containerID="c3982048060959dc568510522d4d52df9e2865fc4d4911535daada38fea131dc" Jul 06 20:15:08.887433 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.887045 2566 scope.go:117] "RemoveContainer" containerID="7a26af0ef24929c62a60edbf6a98b398c801ef6dcf6c3fba0ba4e7774de1ecf6" Jul 06 20:15:08.887433 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:08.887229 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-g6t88_openshift-console-operator(62d6ada4-8960-4e60-a605-9621138b9c8d)\"" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" podUID="62d6ada4-8960-4e60-a605-9621138b9c8d" Jul 06 20:15:08.891138 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.891081 2566 generic.go:356] "Generic (PLEG): container finished" podID="12b0e04f-1230-4820-a7ce-93a5ce83ef68" containerID="43e1de31756e3f362703a5c5b45679b6fb1aa39b5fdde5094bb320fa5b00f31a" exitCode=0 Jul 06 20:15:08.891138 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.891116 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb74l" event={"ID":"12b0e04f-1230-4820-a7ce-93a5ce83ef68","Type":"ContainerDied","Data":"43e1de31756e3f362703a5c5b45679b6fb1aa39b5fdde5094bb320fa5b00f31a"} Jul 06 20:15:08.967311 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.967287 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw"] Jul 06 20:15:08.999643 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.999606 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw"] Jul 06 20:15:08.999745 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:08.999732 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw" Jul 06 20:15:09.004072 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.004049 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 20:15:09.004612 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.004583 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 20:15:09.004800 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.004645 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kbx8c\"" Jul 06 20:15:09.185670 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.185592 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tffl\" (UniqueName: \"kubernetes.io/projected/900867c2-1a0f-448d-9909-5c0cf59a161a-kube-api-access-9tffl\") pod \"migrator-d57d58cc9-snzgw\" (UID: \"900867c2-1a0f-448d-9909-5c0cf59a161a\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw" Jul 06 20:15:09.286768 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.286738 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tffl\" (UniqueName: \"kubernetes.io/projected/900867c2-1a0f-448d-9909-5c0cf59a161a-kube-api-access-9tffl\") pod \"migrator-d57d58cc9-snzgw\" (UID: \"900867c2-1a0f-448d-9909-5c0cf59a161a\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw" Jul 06 20:15:09.298934 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.298905 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tffl\" (UniqueName: \"kubernetes.io/projected/900867c2-1a0f-448d-9909-5c0cf59a161a-kube-api-access-9tffl\") pod \"migrator-d57d58cc9-snzgw\" (UID: \"900867c2-1a0f-448d-9909-5c0cf59a161a\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw" Jul 06 20:15:09.319902 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.319879 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw" Jul 06 20:15:09.456283 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.456218 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw"] Jul 06 20:15:09.459185 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:09.459163 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900867c2_1a0f_448d_9909_5c0cf59a161a.slice/crio-69a725ba1956f3c4e54585f0b17caf55098b7ce1f9b48ac9f33613ffc41af8a7 WatchSource:0}: Error finding container 69a725ba1956f3c4e54585f0b17caf55098b7ce1f9b48ac9f33613ffc41af8a7: Status 404 returned error can't find the container with id 69a725ba1956f3c4e54585f0b17caf55098b7ce1f9b48ac9f33613ffc41af8a7 Jul 06 20:15:09.896417 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.896384 2566 generic.go:356] "Generic (PLEG): container finished" podID="12b0e04f-1230-4820-a7ce-93a5ce83ef68" containerID="61f31c4648b66b17f1861652aa334f544930f11945744405485da19df65a2e90" exitCode=0 Jul 06 20:15:09.896837 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.896465 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb74l" event={"ID":"12b0e04f-1230-4820-a7ce-93a5ce83ef68","Type":"ContainerDied","Data":"61f31c4648b66b17f1861652aa334f544930f11945744405485da19df65a2e90"} Jul 06 20:15:09.897969 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.897953 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:15:09.898292 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.898272 2566 scope.go:117] "RemoveContainer" containerID="7a26af0ef24929c62a60edbf6a98b398c801ef6dcf6c3fba0ba4e7774de1ecf6" Jul 06 20:15:09.898461 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:09.898434 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-g6t88_openshift-console-operator(62d6ada4-8960-4e60-a605-9621138b9c8d)\"" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" podUID="62d6ada4-8960-4e60-a605-9621138b9c8d" Jul 06 20:15:09.899108 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:09.899092 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw" event={"ID":"900867c2-1a0f-448d-9909-5c0cf59a161a","Type":"ContainerStarted","Data":"69a725ba1956f3c4e54585f0b17caf55098b7ce1f9b48ac9f33613ffc41af8a7"} Jul 06 20:15:10.906059 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:10.906018 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hb74l" event={"ID":"12b0e04f-1230-4820-a7ce-93a5ce83ef68","Type":"ContainerStarted","Data":"494951f8f78eee6b3fc8c382c1791253337e8f6411187c68061c54d63e1e0f30"} Jul 06 20:15:10.938306 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:10.938260 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hb74l" podStartSLOduration=8.439091112 podStartE2EDuration="46.938243997s" totalImagesPullingTime="38.499152885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:14:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:10.936680892 +0000 UTC m=+47.782545593" watchObservedRunningTime="2026-07-06 20:15:10.938243997 +0000 UTC m=+47.784108702" Jul 06 20:15:11.343400 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.343366 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-w2n8j"] Jul 06 20:15:11.360300 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.360269 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-w2n8j"] Jul 06 20:15:11.360428 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.360362 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-w2n8j" Jul 06 20:15:11.363748 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.363569 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 20:15:11.363748 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.363615 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 20:15:11.363952 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.363850 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 20:15:11.364025 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.364004 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7d82g\"" Jul 06 20:15:11.394538 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.394515 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 20:15:11.505377 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.505346 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-245n8\" (UniqueName: \"kubernetes.io/projected/2306e596-47df-4ae7-92c8-e312ec9b1541-kube-api-access-245n8\") pod \"service-ca-758f599c5-w2n8j\" (UID: \"2306e596-47df-4ae7-92c8-e312ec9b1541\") " pod="openshift-service-ca/service-ca-758f599c5-w2n8j" Jul 06 20:15:11.505539 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.505456 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2306e596-47df-4ae7-92c8-e312ec9b1541-signing-cabundle\") pod \"service-ca-758f599c5-w2n8j\" (UID: \"2306e596-47df-4ae7-92c8-e312ec9b1541\") " pod="openshift-service-ca/service-ca-758f599c5-w2n8j" Jul 06 20:15:11.505630 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.505591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2306e596-47df-4ae7-92c8-e312ec9b1541-signing-key\") pod \"service-ca-758f599c5-w2n8j\" (UID: \"2306e596-47df-4ae7-92c8-e312ec9b1541\") " pod="openshift-service-ca/service-ca-758f599c5-w2n8j" Jul 06 20:15:11.606982 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.606908 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-245n8\" (UniqueName: \"kubernetes.io/projected/2306e596-47df-4ae7-92c8-e312ec9b1541-kube-api-access-245n8\") pod \"service-ca-758f599c5-w2n8j\" (UID: \"2306e596-47df-4ae7-92c8-e312ec9b1541\") " pod="openshift-service-ca/service-ca-758f599c5-w2n8j" Jul 06 20:15:11.607110 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.607068 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2306e596-47df-4ae7-92c8-e312ec9b1541-signing-cabundle\") pod \"service-ca-758f599c5-w2n8j\" (UID: \"2306e596-47df-4ae7-92c8-e312ec9b1541\") " pod="openshift-service-ca/service-ca-758f599c5-w2n8j" Jul 06 20:15:11.607168 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.607154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2306e596-47df-4ae7-92c8-e312ec9b1541-signing-key\") pod \"service-ca-758f599c5-w2n8j\" (UID: \"2306e596-47df-4ae7-92c8-e312ec9b1541\") " pod="openshift-service-ca/service-ca-758f599c5-w2n8j" Jul 06 20:15:11.607702 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.607680 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2306e596-47df-4ae7-92c8-e312ec9b1541-signing-cabundle\") pod \"service-ca-758f599c5-w2n8j\" (UID: \"2306e596-47df-4ae7-92c8-e312ec9b1541\") " pod="openshift-service-ca/service-ca-758f599c5-w2n8j" Jul 06 20:15:11.609479 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.609452 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2306e596-47df-4ae7-92c8-e312ec9b1541-signing-key\") pod \"service-ca-758f599c5-w2n8j\" (UID: \"2306e596-47df-4ae7-92c8-e312ec9b1541\") " pod="openshift-service-ca/service-ca-758f599c5-w2n8j" Jul 06 20:15:11.615971 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.615946 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-245n8\" (UniqueName: \"kubernetes.io/projected/2306e596-47df-4ae7-92c8-e312ec9b1541-kube-api-access-245n8\") pod \"service-ca-758f599c5-w2n8j\" (UID: \"2306e596-47df-4ae7-92c8-e312ec9b1541\") " pod="openshift-service-ca/service-ca-758f599c5-w2n8j" Jul 06 20:15:11.672219 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.672196 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-w2n8j" Jul 06 20:15:11.990949 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:11.990909 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-w2n8j"] Jul 06 20:15:11.994462 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:11.994433 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2306e596_47df_4ae7_92c8_e312ec9b1541.slice/crio-662e66ea7637e5e70b9639b111ba94c2bfd7c6cfd24b935bc0a411bdd2d9a250 WatchSource:0}: Error finding container 662e66ea7637e5e70b9639b111ba94c2bfd7c6cfd24b935bc0a411bdd2d9a250: Status 404 returned error can't find the container with id 662e66ea7637e5e70b9639b111ba94c2bfd7c6cfd24b935bc0a411bdd2d9a250 Jul 06 20:15:12.918576 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:12.918531 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw" event={"ID":"900867c2-1a0f-448d-9909-5c0cf59a161a","Type":"ContainerStarted","Data":"65e43c6da21ee2930be7ac9eda017d5dcdad057d422b2a0174268e8c04571af3"} Jul 06 20:15:12.918739 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:12.918606 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw" event={"ID":"900867c2-1a0f-448d-9909-5c0cf59a161a","Type":"ContainerStarted","Data":"764598ca2707804604e248ec013de12496918eb900a19394b580904730a7c057"} Jul 06 20:15:12.919959 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:12.919929 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-w2n8j" event={"ID":"2306e596-47df-4ae7-92c8-e312ec9b1541","Type":"ContainerStarted","Data":"f10087380ccf34e6802a9fdcdf39d398d9364e8bb7b14dbdae8a63209faf9b6b"} Jul 06 20:15:12.920050 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:12.919979 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-w2n8j" event={"ID":"2306e596-47df-4ae7-92c8-e312ec9b1541","Type":"ContainerStarted","Data":"662e66ea7637e5e70b9639b111ba94c2bfd7c6cfd24b935bc0a411bdd2d9a250"} Jul 06 20:15:12.952347 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:12.952303 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-snzgw" podStartSLOduration=2.546499578 podStartE2EDuration="4.952289433s" totalImagesPullingTime="2.405789855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:12.951943702 +0000 UTC m=+49.797808425" watchObservedRunningTime="2026-07-06 20:15:12.952289433 +0000 UTC m=+49.798154138" Jul 06 20:15:12.974394 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:12.974353 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-w2n8j" podStartSLOduration=1.9743439550000002 podStartE2EDuration="1.974343955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:12.974228713 +0000 UTC m=+49.820093428" watchObservedRunningTime="2026-07-06 20:15:12.974343955 +0000 UTC m=+49.820208659" Jul 06 20:15:13.826500 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:13.826465 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:15:13.826967 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:13.826607 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:15:13.826967 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:13.826627 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c4ddc6c69-6q7pm: secret "image-registry-tls" not found Jul 06 20:15:13.826967 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:13.826686 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls podName:a2188220-020e-4d49-95c4-a14e2aba5d35 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:29.826667247 +0000 UTC m=+66.672531945 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls") pod "image-registry-7c4ddc6c69-6q7pm" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35") : secret "image-registry-tls" not found Jul 06 20:15:13.927129 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:13.927098 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:15:13.927306 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:13.927214 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l4dz\" (UID: \"40e2374e-aadf-48c9-8fe8-808402f09282\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:15:13.927306 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:13.927232 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:15:13.927306 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:13.927294 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert podName:81143b00-2314-4ecf-ac5e-fd92e429639c nodeName:}" failed. No retries permitted until 2026-07-06 20:15:29.927274574 +0000 UTC m=+66.773139277 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-zqtxq" (UID: "81143b00-2314-4ecf-ac5e-fd92e429639c") : secret "networking-console-plugin-cert" not found Jul 06 20:15:13.930133 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:13.930100 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/40e2374e-aadf-48c9-8fe8-808402f09282-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-4l4dz\" (UID: \"40e2374e-aadf-48c9-8fe8-808402f09282\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:15:13.984228 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:13.984197 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" Jul 06 20:15:14.029059 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:14.029021 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:15:14.029581 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:14.029520 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:15:14.029692 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:14.029610 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls podName:e23f0332-a4f0-4638-8ac3-a9d89fa3564a nodeName:}" failed. No retries permitted until 2026-07-06 20:15:30.029586717 +0000 UTC m=+66.875451416 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls") pod "dns-default-x27pv" (UID: "e23f0332-a4f0-4638-8ac3-a9d89fa3564a") : secret "dns-default-metrics-tls" not found Jul 06 20:15:14.139854 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:14.139785 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz"] Jul 06 20:15:14.928450 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:14.928416 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" event={"ID":"40e2374e-aadf-48c9-8fe8-808402f09282","Type":"ContainerStarted","Data":"b28dfe8429e810e4d9f735d9c573e5b74c7f142cab25fa78c865a8e76b3ab21c"} Jul 06 20:15:16.938790 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:16.938750 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" event={"ID":"40e2374e-aadf-48c9-8fe8-808402f09282","Type":"ContainerStarted","Data":"62f741467b2718f9036e113b98be0d3c9f2268474316b88109187a872a939e91"} Jul 06 20:15:16.939165 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:16.938808 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" event={"ID":"40e2374e-aadf-48c9-8fe8-808402f09282","Type":"ContainerStarted","Data":"2d55de698de8cfa49195bc5dbaced06406aaca13e1e68fe08e5da4e21c3ba181"} Jul 06 20:15:18.370766 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:18.370733 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:15:18.371204 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:18.370784 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:15:18.371204 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:18.371109 2566 scope.go:117] "RemoveContainer" containerID="7a26af0ef24929c62a60edbf6a98b398c801ef6dcf6c3fba0ba4e7774de1ecf6" Jul 06 20:15:18.947122 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:18.947097 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:15:18.947276 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:18.947166 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" event={"ID":"62d6ada4-8960-4e60-a605-9621138b9c8d","Type":"ContainerStarted","Data":"82643cbed74c4d7f01d58309209240e70e5dcf94967080060d2a5bae7ac89eaf"} Jul 06 20:15:18.947465 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:18.947444 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:15:19.020069 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:19.020017 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-4l4dz" podStartSLOduration=43.818212397 podStartE2EDuration="46.020002174s" totalImagesPullingTime="2.201789777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:16.959031991 +0000 UTC m=+53.804896720" watchObservedRunningTime="2026-07-06 20:15:19.020002174 +0000 UTC m=+55.865866879" Jul 06 20:15:19.072362 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:19.072341 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" Jul 06 20:15:19.111817 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:19.111779 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-g6t88" podStartSLOduration=37.261276835 podStartE2EDuration="46.111765861s" totalImagesPullingTime="8.850489026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:19.031210002 +0000 UTC m=+55.877074707" watchObservedRunningTime="2026-07-06 20:15:19.111765861 +0000 UTC m=+55.957630563" Jul 06 20:15:23.808237 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:23.808211 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfhzw" Jul 06 20:15:28.438769 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.438727 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:15:28.441507 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.441487 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 20:15:28.451350 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.451325 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e865c7c9-be01-4556-9120-f629f6a64fc0-metrics-certs\") pod \"network-metrics-daemon-r4xhr\" (UID: \"e865c7c9-be01-4556-9120-f629f6a64fc0\") " pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:15:28.513302 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.513276 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ksdkk\"" Jul 06 20:15:28.521189 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.521173 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4xhr" Jul 06 20:15:28.539340 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.539318 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:15:28.539441 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.539359 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvd9\" (UniqueName: \"kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9\") pod \"network-check-target-5slw5\" (UID: \"812aa8ab-4982-4bba-9655-a5d69680cc8b\") " pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:15:28.541917 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.541895 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 20:15:28.542032 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.541897 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twvd9\" (UniqueName: \"kubernetes.io/projected/812aa8ab-4982-4bba-9655-a5d69680cc8b-kube-api-access-twvd9\") pod \"network-check-target-5slw5\" (UID: \"812aa8ab-4982-4bba-9655-a5d69680cc8b\") " pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:15:28.551522 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.551500 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/985b8c24-44e1-435e-96c0-e99a601b96c1-original-pull-secret\") pod \"global-pull-secret-syncer-tjmw4\" (UID: \"985b8c24-44e1-435e-96c0-e99a601b96c1\") " pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:15:28.643668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.643633 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r4xhr"] Jul 06 20:15:28.646560 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:28.646525 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode865c7c9_be01_4556_9120_f629f6a64fc0.slice/crio-cb0a88218857f33a7856af6018c23d0ace7e50a3728355ea850ee2c67fdb6d4f WatchSource:0}: Error finding container cb0a88218857f33a7856af6018c23d0ace7e50a3728355ea850ee2c67fdb6d4f: Status 404 returned error can't find the container with id cb0a88218857f33a7856af6018c23d0ace7e50a3728355ea850ee2c67fdb6d4f Jul 06 20:15:28.794566 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.794524 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tjmw4" Jul 06 20:15:28.804907 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.804891 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8g8p6\"" Jul 06 20:15:28.813129 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.813104 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:15:28.920802 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.920778 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tjmw4"] Jul 06 20:15:28.923255 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:28.923227 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985b8c24_44e1_435e_96c0_e99a601b96c1.slice/crio-6959852dc9d8bee9db292208183de2c3c1dc9907ed969fecad9fea8f4ee27b5d WatchSource:0}: Error finding container 6959852dc9d8bee9db292208183de2c3c1dc9907ed969fecad9fea8f4ee27b5d: Status 404 returned error can't find the container with id 6959852dc9d8bee9db292208183de2c3c1dc9907ed969fecad9fea8f4ee27b5d Jul 06 20:15:28.941808 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.941787 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5slw5"] Jul 06 20:15:28.944415 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:28.944384 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812aa8ab_4982_4bba_9655_a5d69680cc8b.slice/crio-59c735242d5379dfcbd14547ac8103e0cb11efcb1a0ffe9557e8f62d00debdc1 WatchSource:0}: Error finding container 59c735242d5379dfcbd14547ac8103e0cb11efcb1a0ffe9557e8f62d00debdc1: Status 404 returned error can't find the container with id 59c735242d5379dfcbd14547ac8103e0cb11efcb1a0ffe9557e8f62d00debdc1 Jul 06 20:15:28.979801 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.979771 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tjmw4" event={"ID":"985b8c24-44e1-435e-96c0-e99a601b96c1","Type":"ContainerStarted","Data":"6959852dc9d8bee9db292208183de2c3c1dc9907ed969fecad9fea8f4ee27b5d"} Jul 06 20:15:28.981309 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.981272 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4xhr" event={"ID":"e865c7c9-be01-4556-9120-f629f6a64fc0","Type":"ContainerStarted","Data":"cb0a88218857f33a7856af6018c23d0ace7e50a3728355ea850ee2c67fdb6d4f"} Jul 06 20:15:28.982485 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:28.982465 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5slw5" event={"ID":"812aa8ab-4982-4bba-9655-a5d69680cc8b","Type":"ContainerStarted","Data":"59c735242d5379dfcbd14547ac8103e0cb11efcb1a0ffe9557e8f62d00debdc1"} Jul 06 20:15:29.851469 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:29.851436 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:15:29.854217 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:29.854170 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls\") pod \"image-registry-7c4ddc6c69-6q7pm\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:15:29.952440 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:29.952413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:15:29.955512 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:29.955453 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/81143b00-2314-4ecf-ac5e-fd92e429639c-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-zqtxq\" (UID: \"81143b00-2314-4ecf-ac5e-fd92e429639c\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:15:29.986838 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:29.986810 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5slw5" event={"ID":"812aa8ab-4982-4bba-9655-a5d69680cc8b","Type":"ContainerStarted","Data":"279c82e9e38d903a53bfa80def15c19e6e771854ff99b50c563240eca17ec4dd"} Jul 06 20:15:29.986969 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:29.986876 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:15:29.988471 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:29.988445 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4xhr" event={"ID":"e865c7c9-be01-4556-9120-f629f6a64fc0","Type":"ContainerStarted","Data":"df95e72eb90eccf33dc33e571445eb7df3d47f4eebb9ecaacf06ee618983d0ca"} Jul 06 20:15:29.988589 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:29.988489 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4xhr" event={"ID":"e865c7c9-be01-4556-9120-f629f6a64fc0","Type":"ContainerStarted","Data":"459b214314812d1ee38a658f50e442b0f4a54701280b1dd5f3f750e0a374dbb3"} Jul 06 20:15:30.003969 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.003693 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5slw5" podStartSLOduration=66.003675422 podStartE2EDuration="1m6.003675422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:30.003381774 +0000 UTC m=+66.849246477" watchObservedRunningTime="2026-07-06 20:15:30.003675422 +0000 UTC m=+66.849540126" Jul 06 20:15:30.019499 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.019445 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r4xhr" podStartSLOduration=65.895630253 podStartE2EDuration="1m7.01943228s" totalImagesPullingTime="1.123802027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:30.018352142 +0000 UTC m=+66.864216851" watchObservedRunningTime="2026-07-06 20:15:30.01943228 +0000 UTC m=+66.865296982" Jul 06 20:15:30.052995 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.052935 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:15:30.055966 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.055945 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e23f0332-a4f0-4638-8ac3-a9d89fa3564a-metrics-tls\") pod \"dns-default-x27pv\" (UID: \"e23f0332-a4f0-4638-8ac3-a9d89fa3564a\") " pod="openshift-dns/dns-default-x27pv" Jul 06 20:15:30.118579 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.118540 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xfxxg\"" Jul 06 20:15:30.126600 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.126581 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:15:30.210284 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.210177 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7kdgf\"" Jul 06 20:15:30.216271 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.216246 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" Jul 06 20:15:30.230996 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.230975 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hk74q\"" Jul 06 20:15:30.238983 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.238957 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x27pv" Jul 06 20:15:30.304713 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.304592 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c4ddc6c69-6q7pm"] Jul 06 20:15:30.309180 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:30.309154 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2188220_020e_4d49_95c4_a14e2aba5d35.slice/crio-6724ab54c9cb2563333b2dd4859cef5e4171d884a07c61a99b0a92875afacc5d WatchSource:0}: Error finding container 6724ab54c9cb2563333b2dd4859cef5e4171d884a07c61a99b0a92875afacc5d: Status 404 returned error can't find the container with id 6724ab54c9cb2563333b2dd4859cef5e4171d884a07c61a99b0a92875afacc5d Jul 06 20:15:30.368677 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.368094 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq"] Jul 06 20:15:30.371405 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:30.371374 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81143b00_2314_4ecf_ac5e_fd92e429639c.slice/crio-1949ca85d698184da12932b3cd05dc77bbdcd0193dc5c84ed4791e9ac2681186 WatchSource:0}: Error finding container 1949ca85d698184da12932b3cd05dc77bbdcd0193dc5c84ed4791e9ac2681186: Status 404 returned error can't find the container with id 1949ca85d698184da12932b3cd05dc77bbdcd0193dc5c84ed4791e9ac2681186 Jul 06 20:15:30.393028 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.393000 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x27pv"] Jul 06 20:15:30.396246 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:30.396223 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode23f0332_a4f0_4638_8ac3_a9d89fa3564a.slice/crio-10cd80a7b6a8dac804a012cc8cb4658b18eba6b5a75e160bd102e2695a89e849 WatchSource:0}: Error finding container 10cd80a7b6a8dac804a012cc8cb4658b18eba6b5a75e160bd102e2695a89e849: Status 404 returned error can't find the container with id 10cd80a7b6a8dac804a012cc8cb4658b18eba6b5a75e160bd102e2695a89e849 Jul 06 20:15:30.992712 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.992674 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x27pv" event={"ID":"e23f0332-a4f0-4638-8ac3-a9d89fa3564a","Type":"ContainerStarted","Data":"10cd80a7b6a8dac804a012cc8cb4658b18eba6b5a75e160bd102e2695a89e849"} Jul 06 20:15:30.994178 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.994151 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" event={"ID":"a2188220-020e-4d49-95c4-a14e2aba5d35","Type":"ContainerStarted","Data":"24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534"} Jul 06 20:15:30.994298 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.994203 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:15:30.994298 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.994224 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" event={"ID":"a2188220-020e-4d49-95c4-a14e2aba5d35","Type":"ContainerStarted","Data":"6724ab54c9cb2563333b2dd4859cef5e4171d884a07c61a99b0a92875afacc5d"} Jul 06 20:15:30.995284 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:30.995260 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" event={"ID":"81143b00-2314-4ecf-ac5e-fd92e429639c","Type":"ContainerStarted","Data":"1949ca85d698184da12932b3cd05dc77bbdcd0193dc5c84ed4791e9ac2681186"} Jul 06 20:15:31.017022 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:31.016981 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" podStartSLOduration=67.016968276 podStartE2EDuration="1m7.016968276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:31.01643423 +0000 UTC m=+67.862298935" watchObservedRunningTime="2026-07-06 20:15:31.016968276 +0000 UTC m=+67.862832981" Jul 06 20:15:32.000302 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:32.000247 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" event={"ID":"81143b00-2314-4ecf-ac5e-fd92e429639c","Type":"ContainerStarted","Data":"660f15d8ee35c3b9b314c6637486f327958924a89c8c1828b65da7a04d7bed26"} Jul 06 20:15:32.018499 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:32.018427 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-zqtxq" podStartSLOduration=35.508685235 podStartE2EDuration="37.018406476s" totalImagesPullingTime="1.509721241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:32.0172272 +0000 UTC m=+68.863091904" watchObservedRunningTime="2026-07-06 20:15:32.018406476 +0000 UTC m=+68.864271182" Jul 06 20:15:33.428850 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.428779 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct"] Jul 06 20:15:33.431280 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.431256 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5"] Jul 06 20:15:33.431403 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.431367 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" Jul 06 20:15:33.433797 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.433772 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h"] Jul 06 20:15:33.433931 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.433884 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:33.434909 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.434891 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 20:15:33.435013 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.434894 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 20:15:33.435779 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.435752 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 20:15:33.436023 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.436007 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c"] Jul 06 20:15:33.436129 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.436115 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h" Jul 06 20:15:33.436757 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.436741 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6gpnd\"" Jul 06 20:15:33.436861 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.436810 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 20:15:33.437932 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.437896 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 20:15:33.438027 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.437905 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 20:15:33.438123 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.438105 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 20:15:33.438236 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.438220 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.440013 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.439998 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 20:15:33.440492 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.440451 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-dmc8x\"" Jul 06 20:15:33.440721 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.440707 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 06 20:15:33.440798 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.440744 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 06 20:15:33.441441 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.441424 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 06 20:15:33.441534 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.441518 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 20:15:33.441663 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.441645 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 06 20:15:33.444271 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.444250 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5"] Jul 06 20:15:33.456038 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.455664 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct"] Jul 06 20:15:33.460875 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.460851 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c"] Jul 06 20:15:33.476873 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.476851 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h"] Jul 06 20:15:33.479114 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479086 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4w2q\" (UniqueName: \"kubernetes.io/projected/8823196a-ede8-4dd3-a54c-e059bd003283-kube-api-access-b4w2q\") pod \"cluster-monitoring-operator-64786d9cbb-r85ct\" (UID: \"8823196a-ede8-4dd3-a54c-e059bd003283\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" Jul 06 20:15:33.479114 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqxx7\" (UniqueName: \"kubernetes.io/projected/7c96825e-d1c0-4874-a214-8589273d151c-kube-api-access-hqxx7\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479221 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-hub\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479281 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8823196a-ede8-4dd3-a54c-e059bd003283-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-r85ct\" (UID: \"8823196a-ede8-4dd3-a54c-e059bd003283\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479317 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7160cb-ba76-4de6-bc6e-0dfc296b556a-tmp\") pod \"klusterlet-addon-workmgr-75f78fbb78-jldb5\" (UID: \"0f7160cb-ba76-4de6-bc6e-0dfc296b556a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-ca\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479358 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7c89a01-41a9-4780-b44a-9ca74c177efe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f6dfc6666-c4s7h\" (UID: \"b7c89a01-41a9-4780-b44a-9ca74c177efe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479382 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8823196a-ede8-4dd3-a54c-e059bd003283-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-r85ct\" (UID: \"8823196a-ede8-4dd3-a54c-e059bd003283\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt2cj\" (UniqueName: \"kubernetes.io/projected/0f7160cb-ba76-4de6-bc6e-0dfc296b556a-kube-api-access-gt2cj\") pod \"klusterlet-addon-workmgr-75f78fbb78-jldb5\" (UID: \"0f7160cb-ba76-4de6-bc6e-0dfc296b556a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479427 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7pz7\" (UniqueName: \"kubernetes.io/projected/b7c89a01-41a9-4780-b44a-9ca74c177efe-kube-api-access-t7pz7\") pod \"managed-serviceaccount-addon-agent-f6dfc6666-c4s7h\" (UID: \"b7c89a01-41a9-4780-b44a-9ca74c177efe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479456 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479482 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c96825e-d1c0-4874-a214-8589273d151c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.479598 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.479519 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0f7160cb-ba76-4de6-bc6e-0dfc296b556a-klusterlet-config\") pod \"klusterlet-addon-workmgr-75f78fbb78-jldb5\" (UID: \"0f7160cb-ba76-4de6-bc6e-0dfc296b556a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:33.534707 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.534664 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-g9qjv"] Jul 06 20:15:33.537633 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.537610 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-g9qjv" Jul 06 20:15:33.537755 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.537615 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-57bb47759f-dq5zh"] Jul 06 20:15:33.542509 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.540341 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.542509 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.540826 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 20:15:33.542509 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.540961 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-crnhw\"" Jul 06 20:15:33.542509 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.542363 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 20:15:33.543228 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.543209 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-pwjb2"] Jul 06 20:15:33.546215 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.546190 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.546837 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.546817 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 20:15:33.546944 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.546910 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-p9dnb\"" Jul 06 20:15:33.547008 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.546962 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 20:15:33.547640 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.547623 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 20:15:33.550357 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.550336 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 20:15:33.550465 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.550421 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 20:15:33.550561 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.550526 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 20:15:33.556873 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.556853 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5z82h\"" Jul 06 20:15:33.557274 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.557254 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 20:15:33.558439 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.558418 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 20:15:33.558557 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.558504 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 20:15:33.558743 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.558727 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 20:15:33.564974 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.564956 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-g9qjv"] Jul 06 20:15:33.565978 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.565957 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 20:15:33.568239 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.568220 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57bb47759f-dq5zh"] Jul 06 20:15:33.578102 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.578081 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-pwjb2"] Jul 06 20:15:33.580009 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.579985 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faff23fe-0eb2-45d5-a5ea-85f830547761-metrics-certs\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.580111 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580041 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.580111 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-hub\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.580238 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580152 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8823196a-ede8-4dd3-a54c-e059bd003283-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-r85ct\" (UID: \"8823196a-ede8-4dd3-a54c-e059bd003283\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" Jul 06 20:15:33.580238 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580186 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7160cb-ba76-4de6-bc6e-0dfc296b556a-tmp\") pod \"klusterlet-addon-workmgr-75f78fbb78-jldb5\" (UID: \"0f7160cb-ba76-4de6-bc6e-0dfc296b556a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:33.580238 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580211 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-ca\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.580400 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580241 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/faff23fe-0eb2-45d5-a5ea-85f830547761-stats-auth\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.580400 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580268 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7c89a01-41a9-4780-b44a-9ca74c177efe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f6dfc6666-c4s7h\" (UID: \"b7c89a01-41a9-4780-b44a-9ca74c177efe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h" Jul 06 20:15:33.580400 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580299 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8823196a-ede8-4dd3-a54c-e059bd003283-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-r85ct\" (UID: \"8823196a-ede8-4dd3-a54c-e059bd003283\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" Jul 06 20:15:33.580400 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580330 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt2cj\" (UniqueName: \"kubernetes.io/projected/0f7160cb-ba76-4de6-bc6e-0dfc296b556a-kube-api-access-gt2cj\") pod \"klusterlet-addon-workmgr-75f78fbb78-jldb5\" (UID: \"0f7160cb-ba76-4de6-bc6e-0dfc296b556a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:33.580400 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580358 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7pz7\" (UniqueName: \"kubernetes.io/projected/b7c89a01-41a9-4780-b44a-9ca74c177efe-kube-api-access-t7pz7\") pod \"managed-serviceaccount-addon-agent-f6dfc6666-c4s7h\" (UID: \"b7c89a01-41a9-4780-b44a-9ca74c177efe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h" Jul 06 20:15:33.580400 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580387 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmxw6\" (UniqueName: \"kubernetes.io/projected/faff23fe-0eb2-45d5-a5ea-85f830547761-kube-api-access-rmxw6\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.580720 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580413 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfhtv\" (UniqueName: \"kubernetes.io/projected/0f0c847d-119c-469b-943d-e385d909b9b3-kube-api-access-xfhtv\") pod \"downloads-685bfb5f64-g9qjv\" (UID: \"0f0c847d-119c-469b-943d-e385d909b9b3\") " pod="openshift-console/downloads-685bfb5f64-g9qjv" Jul 06 20:15:33.580720 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580449 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.580720 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580477 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faff23fe-0eb2-45d5-a5ea-85f830547761-service-ca-bundle\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.580720 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c96825e-d1c0-4874-a214-8589273d151c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.580720 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580541 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0f7160cb-ba76-4de6-bc6e-0dfc296b556a-klusterlet-config\") pod \"klusterlet-addon-workmgr-75f78fbb78-jldb5\" (UID: \"0f7160cb-ba76-4de6-bc6e-0dfc296b556a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:33.580720 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580597 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/faff23fe-0eb2-45d5-a5ea-85f830547761-default-certificate\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.580720 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580627 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4w2q\" (UniqueName: \"kubernetes.io/projected/8823196a-ede8-4dd3-a54c-e059bd003283-kube-api-access-b4w2q\") pod \"cluster-monitoring-operator-64786d9cbb-r85ct\" (UID: \"8823196a-ede8-4dd3-a54c-e059bd003283\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" Jul 06 20:15:33.580720 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.580658 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqxx7\" (UniqueName: \"kubernetes.io/projected/7c96825e-d1c0-4874-a214-8589273d151c-kube-api-access-hqxx7\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.582254 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.581645 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c96825e-d1c0-4874-a214-8589273d151c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.582254 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.582184 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7160cb-ba76-4de6-bc6e-0dfc296b556a-tmp\") pod \"klusterlet-addon-workmgr-75f78fbb78-jldb5\" (UID: \"0f7160cb-ba76-4de6-bc6e-0dfc296b556a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:33.584584 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.583807 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8823196a-ede8-4dd3-a54c-e059bd003283-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-r85ct\" (UID: \"8823196a-ede8-4dd3-a54c-e059bd003283\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" Jul 06 20:15:33.584584 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.583944 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.584584 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.584028 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7c89a01-41a9-4780-b44a-9ca74c177efe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f6dfc6666-c4s7h\" (UID: \"b7c89a01-41a9-4780-b44a-9ca74c177efe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h" Jul 06 20:15:33.584584 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.584304 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-hub\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.585088 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.585048 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.585745 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.585702 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0f7160cb-ba76-4de6-bc6e-0dfc296b556a-klusterlet-config\") pod \"klusterlet-addon-workmgr-75f78fbb78-jldb5\" (UID: \"0f7160cb-ba76-4de6-bc6e-0dfc296b556a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:33.585900 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.585883 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8823196a-ede8-4dd3-a54c-e059bd003283-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-r85ct\" (UID: \"8823196a-ede8-4dd3-a54c-e059bd003283\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" Jul 06 20:15:33.586200 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.586185 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7c96825e-d1c0-4874-a214-8589273d151c-ca\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.608896 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.608865 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqxx7\" (UniqueName: \"kubernetes.io/projected/7c96825e-d1c0-4874-a214-8589273d151c-kube-api-access-hqxx7\") pod \"cluster-proxy-proxy-agent-57b55c565f-hvn9c\" (UID: \"7c96825e-d1c0-4874-a214-8589273d151c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.611949 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.611907 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4w2q\" (UniqueName: \"kubernetes.io/projected/8823196a-ede8-4dd3-a54c-e059bd003283-kube-api-access-b4w2q\") pod \"cluster-monitoring-operator-64786d9cbb-r85ct\" (UID: \"8823196a-ede8-4dd3-a54c-e059bd003283\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" Jul 06 20:15:33.621065 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.621026 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7pz7\" (UniqueName: \"kubernetes.io/projected/b7c89a01-41a9-4780-b44a-9ca74c177efe-kube-api-access-t7pz7\") pod \"managed-serviceaccount-addon-agent-f6dfc6666-c4s7h\" (UID: \"b7c89a01-41a9-4780-b44a-9ca74c177efe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h" Jul 06 20:15:33.628962 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.628941 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt2cj\" (UniqueName: \"kubernetes.io/projected/0f7160cb-ba76-4de6-bc6e-0dfc296b556a-kube-api-access-gt2cj\") pod \"klusterlet-addon-workmgr-75f78fbb78-jldb5\" (UID: \"0f7160cb-ba76-4de6-bc6e-0dfc296b556a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:33.681528 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681452 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxw6\" (UniqueName: \"kubernetes.io/projected/faff23fe-0eb2-45d5-a5ea-85f830547761-kube-api-access-rmxw6\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.681528 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681490 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfhtv\" (UniqueName: \"kubernetes.io/projected/0f0c847d-119c-469b-943d-e385d909b9b3-kube-api-access-xfhtv\") pod \"downloads-685bfb5f64-g9qjv\" (UID: \"0f0c847d-119c-469b-943d-e385d909b9b3\") " pod="openshift-console/downloads-685bfb5f64-g9qjv" Jul 06 20:15:33.681528 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681521 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faff23fe-0eb2-45d5-a5ea-85f830547761-service-ca-bundle\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.681795 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681582 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39246747-c52b-40a4-b3d5-bf8fed1223dd-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.681795 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681609 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/faff23fe-0eb2-45d5-a5ea-85f830547761-default-certificate\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.681795 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681636 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39246747-c52b-40a4-b3d5-bf8fed1223dd-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.681795 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681672 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwb8w\" (UniqueName: \"kubernetes.io/projected/39246747-c52b-40a4-b3d5-bf8fed1223dd-kube-api-access-qwb8w\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.681795 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681706 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faff23fe-0eb2-45d5-a5ea-85f830547761-metrics-certs\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.681795 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/39246747-c52b-40a4-b3d5-bf8fed1223dd-serving-cert\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.681795 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681779 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39246747-c52b-40a4-b3d5-bf8fed1223dd-tmp\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.682024 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681847 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/39246747-c52b-40a4-b3d5-bf8fed1223dd-snapshots\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.682024 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.681896 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/faff23fe-0eb2-45d5-a5ea-85f830547761-stats-auth\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.682261 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.682242 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faff23fe-0eb2-45d5-a5ea-85f830547761-service-ca-bundle\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.684231 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.684211 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/faff23fe-0eb2-45d5-a5ea-85f830547761-default-certificate\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.684320 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.684248 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faff23fe-0eb2-45d5-a5ea-85f830547761-metrics-certs\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.684397 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.684375 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/faff23fe-0eb2-45d5-a5ea-85f830547761-stats-auth\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.693377 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.693354 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfhtv\" (UniqueName: \"kubernetes.io/projected/0f0c847d-119c-469b-943d-e385d909b9b3-kube-api-access-xfhtv\") pod \"downloads-685bfb5f64-g9qjv\" (UID: \"0f0c847d-119c-469b-943d-e385d909b9b3\") " pod="openshift-console/downloads-685bfb5f64-g9qjv" Jul 06 20:15:33.694138 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.694119 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmxw6\" (UniqueName: \"kubernetes.io/projected/faff23fe-0eb2-45d5-a5ea-85f830547761-kube-api-access-rmxw6\") pod \"router-default-57bb47759f-dq5zh\" (UID: \"faff23fe-0eb2-45d5-a5ea-85f830547761\") " pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.742125 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.742100 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" Jul 06 20:15:33.749673 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.749649 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:33.763501 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.763475 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h" Jul 06 20:15:33.770666 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.770637 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" Jul 06 20:15:33.783237 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.783208 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39246747-c52b-40a4-b3d5-bf8fed1223dd-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.783341 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.783255 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwb8w\" (UniqueName: \"kubernetes.io/projected/39246747-c52b-40a4-b3d5-bf8fed1223dd-kube-api-access-qwb8w\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.783341 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.783287 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/39246747-c52b-40a4-b3d5-bf8fed1223dd-serving-cert\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.783341 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.783321 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39246747-c52b-40a4-b3d5-bf8fed1223dd-tmp\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.783493 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.783363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/39246747-c52b-40a4-b3d5-bf8fed1223dd-snapshots\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.783493 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.783434 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39246747-c52b-40a4-b3d5-bf8fed1223dd-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.784436 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.784125 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39246747-c52b-40a4-b3d5-bf8fed1223dd-tmp\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.784436 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.784263 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39246747-c52b-40a4-b3d5-bf8fed1223dd-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.784613 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.784488 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39246747-c52b-40a4-b3d5-bf8fed1223dd-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.784827 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.784809 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/39246747-c52b-40a4-b3d5-bf8fed1223dd-snapshots\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.787493 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.787464 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/39246747-c52b-40a4-b3d5-bf8fed1223dd-serving-cert\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.798828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.798801 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwb8w\" (UniqueName: \"kubernetes.io/projected/39246747-c52b-40a4-b3d5-bf8fed1223dd-kube-api-access-qwb8w\") pod \"insights-operator-5c847dcfc4-pwjb2\" (UID: \"39246747-c52b-40a4-b3d5-bf8fed1223dd\") " pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.858341 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.858082 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-g9qjv" Jul 06 20:15:33.874998 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.874798 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" Jul 06 20:15:33.875409 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.875334 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:33.962433 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.962286 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct"] Jul 06 20:15:33.982488 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:33.982445 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5"] Jul 06 20:15:34.009233 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:34.009167 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h"] Jul 06 20:15:34.030491 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:34.026332 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x27pv" event={"ID":"e23f0332-a4f0-4638-8ac3-a9d89fa3564a","Type":"ContainerStarted","Data":"69afc37fa37c694b76a33fc5ee3708312a7c1008fc5abd3b22f3332821bc8d57"} Jul 06 20:15:34.030491 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:34.026407 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x27pv" event={"ID":"e23f0332-a4f0-4638-8ac3-a9d89fa3564a","Type":"ContainerStarted","Data":"7d3191e836810e436ef102fd9dbd63073a9410d50860db62a785d4c0542b1f55"} Jul 06 20:15:34.030491 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:34.027252 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x27pv" Jul 06 20:15:34.030491 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:34.029696 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" event={"ID":"8823196a-ede8-4dd3-a54c-e059bd003283","Type":"ContainerStarted","Data":"23388059a4f431fa88b22521332282d52e968a99127a261350fb0bd36168278a"} Jul 06 20:15:34.032364 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:34.032297 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" event={"ID":"0f7160cb-ba76-4de6-bc6e-0dfc296b556a","Type":"ContainerStarted","Data":"c81555854bccc3044f27f649c4092866efcce83bd0b84c980e70aaf720251575"} Jul 06 20:15:34.038682 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:34.038657 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c"] Jul 06 20:15:34.046867 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:34.046447 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x27pv" podStartSLOduration=33.289112418 podStartE2EDuration="36.046429917s" totalImagesPullingTime="2.757317499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:34.046073541 +0000 UTC m=+70.891938247" watchObservedRunningTime="2026-07-06 20:15:34.046429917 +0000 UTC m=+70.892294624" Jul 06 20:15:34.048465 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:34.043407 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c96825e_d1c0_4874_a214_8589273d151c.slice/crio-7b1b1091848b85b3b9f2a9fab9c3be037a995f799b898b317861a7cdf7925c34 WatchSource:0}: Error finding container 7b1b1091848b85b3b9f2a9fab9c3be037a995f799b898b317861a7cdf7925c34: Status 404 returned error can't find the container with id 7b1b1091848b85b3b9f2a9fab9c3be037a995f799b898b317861a7cdf7925c34 Jul 06 20:15:34.083129 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:34.083072 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-g9qjv"] Jul 06 20:15:34.085632 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:34.085607 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0c847d_119c_469b_943d_e385d909b9b3.slice/crio-78674e61069072338d211a8666915d7e20a88d27c940bf1eb817dcf82ff9634b WatchSource:0}: Error finding container 78674e61069072338d211a8666915d7e20a88d27c940bf1eb817dcf82ff9634b: Status 404 returned error can't find the container with id 78674e61069072338d211a8666915d7e20a88d27c940bf1eb817dcf82ff9634b Jul 06 20:15:34.107739 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:34.107713 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-pwjb2"] Jul 06 20:15:34.111124 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:34.111099 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39246747_c52b_40a4_b3d5_bf8fed1223dd.slice/crio-93bfb745e996aaaec5836e9b1b4d0e88b3f65368e01848d84b2d82cb4eb40e88 WatchSource:0}: Error finding container 93bfb745e996aaaec5836e9b1b4d0e88b3f65368e01848d84b2d82cb4eb40e88: Status 404 returned error can't find the container with id 93bfb745e996aaaec5836e9b1b4d0e88b3f65368e01848d84b2d82cb4eb40e88 Jul 06 20:15:34.125596 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:34.125577 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57bb47759f-dq5zh"] Jul 06 20:15:34.127802 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:34.127778 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaff23fe_0eb2_45d5_a5ea_85f830547761.slice/crio-79c411da03dc0554869348997d5305400eee36fc3236ce94de81629fff5db481 WatchSource:0}: Error finding container 79c411da03dc0554869348997d5305400eee36fc3236ce94de81629fff5db481: Status 404 returned error can't find the container with id 79c411da03dc0554869348997d5305400eee36fc3236ce94de81629fff5db481 Jul 06 20:15:35.039063 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:35.039020 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" event={"ID":"39246747-c52b-40a4-b3d5-bf8fed1223dd","Type":"ContainerStarted","Data":"93bfb745e996aaaec5836e9b1b4d0e88b3f65368e01848d84b2d82cb4eb40e88"} Jul 06 20:15:35.040496 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:35.040467 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" event={"ID":"7c96825e-d1c0-4874-a214-8589273d151c","Type":"ContainerStarted","Data":"7b1b1091848b85b3b9f2a9fab9c3be037a995f799b898b317861a7cdf7925c34"} Jul 06 20:15:35.041863 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:35.041835 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h" event={"ID":"b7c89a01-41a9-4780-b44a-9ca74c177efe","Type":"ContainerStarted","Data":"4f888046f5c501cea5b2145be2614060d96ac3cc5dccf07969d90d1c0f3189fe"} Jul 06 20:15:35.043098 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:35.043071 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-g9qjv" event={"ID":"0f0c847d-119c-469b-943d-e385d909b9b3","Type":"ContainerStarted","Data":"78674e61069072338d211a8666915d7e20a88d27c940bf1eb817dcf82ff9634b"} Jul 06 20:15:35.044753 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:35.044728 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57bb47759f-dq5zh" event={"ID":"faff23fe-0eb2-45d5-a5ea-85f830547761","Type":"ContainerStarted","Data":"5e09ddf1fd28282933f108f9ce928c67522b27dd166154104173cc7244a17bfa"} Jul 06 20:15:35.044831 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:35.044774 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57bb47759f-dq5zh" event={"ID":"faff23fe-0eb2-45d5-a5ea-85f830547761","Type":"ContainerStarted","Data":"79c411da03dc0554869348997d5305400eee36fc3236ce94de81629fff5db481"} Jul 06 20:15:35.075774 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:35.075692 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-57bb47759f-dq5zh" podStartSLOduration=2.075675604 podStartE2EDuration="2.075675604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:35.075515737 +0000 UTC m=+71.921380478" watchObservedRunningTime="2026-07-06 20:15:35.075675604 +0000 UTC m=+71.921540310" Jul 06 20:15:35.876083 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:35.876057 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:35.878703 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:35.878677 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:36.056625 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:36.055606 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tjmw4" event={"ID":"985b8c24-44e1-435e-96c0-e99a601b96c1","Type":"ContainerStarted","Data":"c711f679362ac664692ec0da010b8c0d2f93efa4177b6d5767faba4152b73b04"} Jul 06 20:15:36.057400 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:36.057135 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:36.059264 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:36.059180 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-57bb47759f-dq5zh" Jul 06 20:15:36.108315 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:36.106917 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tjmw4" podStartSLOduration=65.255000313 podStartE2EDuration="1m12.106896009s" totalImagesPullingTime="6.851895696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:36.078134144 +0000 UTC m=+72.923998850" watchObservedRunningTime="2026-07-06 20:15:36.106896009 +0000 UTC m=+72.952760717" Jul 06 20:15:44.101993 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.101958 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" event={"ID":"0f7160cb-ba76-4de6-bc6e-0dfc296b556a","Type":"ContainerStarted","Data":"f62e68f8b47d5fdc8a4e39a98ef789dad4bd20e56d870bcd1347507e5c03d8ca"} Jul 06 20:15:44.102959 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.102937 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:44.105184 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.105150 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" event={"ID":"39246747-c52b-40a4-b3d5-bf8fed1223dd","Type":"ContainerStarted","Data":"11e11f21ca768245ab972348b5678004cc056e9f07e5cf02692307d02e2d3dd7"} Jul 06 20:15:44.108052 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.108024 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" event={"ID":"7c96825e-d1c0-4874-a214-8589273d151c","Type":"ContainerStarted","Data":"ed50d054a4ea2c8a70f99ee617adfa67fbc83bad900b71920e694d8c9a178f38"} Jul 06 20:15:44.110657 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.110636 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" Jul 06 20:15:44.114286 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.113835 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" event={"ID":"8823196a-ede8-4dd3-a54c-e059bd003283","Type":"ContainerStarted","Data":"a337e24706f555aa04c10eb420f85ae084ee5e5e7e506d57b29f2fdd13b3a0b6"} Jul 06 20:15:44.125125 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.125001 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75f78fbb78-jldb5" podStartSLOduration=1.573887638 podStartE2EDuration="11.124985935s" totalImagesPullingTime="9.551098297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:44.124293344 +0000 UTC m=+80.970158044" watchObservedRunningTime="2026-07-06 20:15:44.124985935 +0000 UTC m=+80.970850641" Jul 06 20:15:44.224793 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.224696 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-pwjb2" podStartSLOduration=1.804967427 podStartE2EDuration="11.224673288s" totalImagesPullingTime="9.419705861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:44.22283208 +0000 UTC m=+81.068696785" watchObservedRunningTime="2026-07-06 20:15:44.224673288 +0000 UTC m=+81.070537992" Jul 06 20:15:44.224944 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.224863 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-r85ct" podStartSLOduration=1.689628381 podStartE2EDuration="11.224852087s" totalImagesPullingTime="9.535223706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:44.171474908 +0000 UTC m=+81.017339612" watchObservedRunningTime="2026-07-06 20:15:44.224852087 +0000 UTC m=+81.070716793" Jul 06 20:15:44.266693 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.265626 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lvtdc"] Jul 06 20:15:44.284467 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.283907 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.287934 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.287636 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xjp7h\"" Jul 06 20:15:44.288833 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.288652 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 20:15:44.291246 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.291092 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 20:15:44.297440 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.297406 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lvtdc"] Jul 06 20:15:44.389508 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.389482 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-crio-socket\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.389643 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.389523 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6rlc\" (UniqueName: \"kubernetes.io/projected/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-kube-api-access-w6rlc\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.389643 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.389633 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-data-volume\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.389760 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.389663 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.389760 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.389695 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.490447 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.490414 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-crio-socket\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.490645 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.490460 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6rlc\" (UniqueName: \"kubernetes.io/projected/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-kube-api-access-w6rlc\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.490645 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.490514 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-data-volume\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.490645 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.490529 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.490645 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.490562 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.490862 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:44.490660 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 20:15:44.490862 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.490662 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-crio-socket\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.490862 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:44.490661 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:15:44.490862 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:44.490728 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-kube-rbac-proxy-cm podName:748c7f38-4aa4-4279-b7db-b0eb53d4e9e7 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:44.990706304 +0000 UTC m=+81.836571007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lvtdc" (UID: "748c7f38-4aa4-4279-b7db-b0eb53d4e9e7") : configmap "kube-rbac-proxy" not found Jul 06 20:15:44.490862 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:44.490747 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-insights-runtime-extractor-tls podName:748c7f38-4aa4-4279-b7db-b0eb53d4e9e7 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:44.990736433 +0000 UTC m=+81.836601117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lvtdc" (UID: "748c7f38-4aa4-4279-b7db-b0eb53d4e9e7") : secret "insights-runtime-extractor-tls" not found Jul 06 20:15:44.491095 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.490912 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-data-volume\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.502015 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.501989 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6rlc\" (UniqueName: \"kubernetes.io/projected/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-kube-api-access-w6rlc\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.998210 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.993844 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.998210 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.993912 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:44.998210 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:44.994067 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:15:44.998210 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:44.994170 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-insights-runtime-extractor-tls podName:748c7f38-4aa4-4279-b7db-b0eb53d4e9e7 nodeName:}" failed. No retries permitted until 2026-07-06 20:15:45.994112714 +0000 UTC m=+82.839977418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lvtdc" (UID: "748c7f38-4aa4-4279-b7db-b0eb53d4e9e7") : secret "insights-runtime-extractor-tls" not found Jul 06 20:15:44.998210 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:44.994630 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:45.058099 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:45.058071 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x27pv" Jul 06 20:15:45.122860 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:45.122820 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" event={"ID":"7c96825e-d1c0-4874-a214-8589273d151c","Type":"ContainerStarted","Data":"b701585864f910eccd7f1cfdc5ca3abb7f2bf1ad2e9af740fa87c9fcf61abec0"} Jul 06 20:15:45.123296 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:45.122884 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" event={"ID":"7c96825e-d1c0-4874-a214-8589273d151c","Type":"ContainerStarted","Data":"94fc3ce7f0c0ebcc171fbc2e6c85391cc33ba6d9192dfd3450f6b70190ea542a"} Jul 06 20:15:45.141313 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:45.141266 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57b55c565f-hvn9c" podStartSLOduration=2.210261332 podStartE2EDuration="12.14124652s" totalImagesPullingTime="9.930985188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:45.1399572 +0000 UTC m=+81.985821905" watchObservedRunningTime="2026-07-06 20:15:45.14124652 +0000 UTC m=+81.987111226" Jul 06 20:15:46.003527 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:46.003395 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:46.005843 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:46.005818 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/748c7f38-4aa4-4279-b7db-b0eb53d4e9e7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lvtdc\" (UID: \"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7\") " pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:46.121346 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:46.121311 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-zqtxq_81143b00-2314-4ecf-ac5e-fd92e429639c/networking-console-plugin/0.log" Jul 06 20:15:46.131497 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:46.131467 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lvtdc" Jul 06 20:15:46.285838 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:46.285807 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lvtdc"] Jul 06 20:15:46.289107 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:46.289069 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748c7f38_4aa4_4279_b7db_b0eb53d4e9e7.slice/crio-db8be634b372996b8f33dcf8e034268276b1fd51da67691dc5a8a4d1186a1912 WatchSource:0}: Error finding container db8be634b372996b8f33dcf8e034268276b1fd51da67691dc5a8a4d1186a1912: Status 404 returned error can't find the container with id db8be634b372996b8f33dcf8e034268276b1fd51da67691dc5a8a4d1186a1912 Jul 06 20:15:46.299883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:46.299861 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:15:46.501880 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:46.501854 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/2.log" Jul 06 20:15:47.138708 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:47.138650 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lvtdc" event={"ID":"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7","Type":"ContainerStarted","Data":"db8be634b372996b8f33dcf8e034268276b1fd51da67691dc5a8a4d1186a1912"} Jul 06 20:15:48.099731 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:48.099690 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x27pv_e23f0332-a4f0-4638-8ac3-a9d89fa3564a/dns/0.log" Jul 06 20:15:48.300215 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:48.300180 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x27pv_e23f0332-a4f0-4638-8ac3-a9d89fa3564a/kube-rbac-proxy/0.log" Jul 06 20:15:48.499770 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:48.499739 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6q7sk_5d7c76e4-8c48-47ca-9514-6d26ac324460/dns-node-resolver/0.log" Jul 06 20:15:49.301501 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:49.301457 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c4ddc6c69-6q7pm_a2188220-020e-4d49-95c4-a14e2aba5d35/registry/0.log" Jul 06 20:15:49.900053 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:49.900020 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tlhdp_1c6ffb1c-9153-4f2d-83ba-967ed6a026a1/node-ca/0.log" Jul 06 20:15:50.099997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:50.099970 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57bb47759f-dq5zh_faff23fe-0eb2-45d5-a5ea-85f830547761/router/0.log" Jul 06 20:15:50.132636 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:50.132603 2566 patch_prober.go:28] interesting pod/image-registry-7c4ddc6c69-6q7pm container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 20:15:50.132757 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:50.132657 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" podUID="a2188220-020e-4d49-95c4-a14e2aba5d35" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:15:50.299856 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:50.299806 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ckhsf_52e1eff4-f7e7-4307-a72b-2bd3b2e58241/serve-healthcheck-canary/0.log" Jul 06 20:15:52.006508 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:52.006471 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:15:54.626675 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.626343 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2lzrs"] Jul 06 20:15:54.664706 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.664671 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.667204 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.667161 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 20:15:54.667696 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.667530 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 20:15:54.668150 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.668129 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 20:15:54.668378 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.668360 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-97kgv\"" Jul 06 20:15:54.668651 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.668634 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 20:15:54.790617 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.790407 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afd18897-c080-4905-98b3-17b79190b39b-sys\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.790617 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.790464 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-accelerators-collector-config\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.790617 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.790497 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-textfile\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.790617 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.790521 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9z4d\" (UniqueName: \"kubernetes.io/projected/afd18897-c080-4905-98b3-17b79190b39b-kube-api-access-p9z4d\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.791233 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.791014 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-tls\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.791233 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.791055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afd18897-c080-4905-98b3-17b79190b39b-metrics-client-ca\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.791233 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.791092 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/afd18897-c080-4905-98b3-17b79190b39b-root\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.791233 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.791110 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.791233 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.791140 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-wtmp\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.892799 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.892716 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afd18897-c080-4905-98b3-17b79190b39b-metrics-client-ca\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.892924 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.892809 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/afd18897-c080-4905-98b3-17b79190b39b-root\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.892924 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.892849 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.892924 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.892891 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-wtmp\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.893039 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.892937 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afd18897-c080-4905-98b3-17b79190b39b-sys\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.893039 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.892968 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-accelerators-collector-config\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.893039 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.892999 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-textfile\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.893039 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.893032 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9z4d\" (UniqueName: \"kubernetes.io/projected/afd18897-c080-4905-98b3-17b79190b39b-kube-api-access-p9z4d\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.893189 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.893082 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-tls\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.893241 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:54.893224 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 20:15:54.893319 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:15:54.893304 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-tls podName:afd18897-c080-4905-98b3-17b79190b39b nodeName:}" failed. No retries permitted until 2026-07-06 20:15:55.393281518 +0000 UTC m=+92.239146217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-tls") pod "node-exporter-2lzrs" (UID: "afd18897-c080-4905-98b3-17b79190b39b") : secret "node-exporter-tls" not found Jul 06 20:15:54.894122 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.894094 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afd18897-c080-4905-98b3-17b79190b39b-metrics-client-ca\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.894256 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.894171 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/afd18897-c080-4905-98b3-17b79190b39b-root\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.894978 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.894915 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-wtmp\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.894978 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.894957 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afd18897-c080-4905-98b3-17b79190b39b-sys\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.895154 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.895097 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-accelerators-collector-config\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.899645 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.895570 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-textfile\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.899645 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.898904 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:54.909263 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:54.909214 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9z4d\" (UniqueName: \"kubernetes.io/projected/afd18897-c080-4905-98b3-17b79190b39b-kube-api-access-p9z4d\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:55.398860 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:55.398811 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-tls\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:55.401335 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:55.401302 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/afd18897-c080-4905-98b3-17b79190b39b-node-exporter-tls\") pod \"node-exporter-2lzrs\" (UID: \"afd18897-c080-4905-98b3-17b79190b39b\") " pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:55.579474 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:55.579431 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2lzrs" Jul 06 20:15:57.342907 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:57.342871 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c4ddc6c69-6q7pm"] Jul 06 20:15:58.845646 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:15:58.845603 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd18897_c080_4905_98b3_17b79190b39b.slice/crio-41a0a199e3c87bfd10ba48f9b25fb3fed4c49acb570da4ed433787f2acb741c5 WatchSource:0}: Error finding container 41a0a199e3c87bfd10ba48f9b25fb3fed4c49acb570da4ed433787f2acb741c5: Status 404 returned error can't find the container with id 41a0a199e3c87bfd10ba48f9b25fb3fed4c49acb570da4ed433787f2acb741c5 Jul 06 20:15:59.192389 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:59.192308 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2lzrs" event={"ID":"afd18897-c080-4905-98b3-17b79190b39b","Type":"ContainerStarted","Data":"41a0a199e3c87bfd10ba48f9b25fb3fed4c49acb570da4ed433787f2acb741c5"} Jul 06 20:15:59.194281 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:59.194184 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h" event={"ID":"b7c89a01-41a9-4780-b44a-9ca74c177efe","Type":"ContainerStarted","Data":"333fd4705a597239ef40209cbff572ade3e404ce41895f0c9814916b79b0a974"} Jul 06 20:15:59.195608 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:59.195540 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lvtdc" event={"ID":"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7","Type":"ContainerStarted","Data":"8afe742ea1decea786c879b6faa2a34225c561ec5a217c836bebca583c2d9f18"} Jul 06 20:15:59.197607 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:59.197176 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-g9qjv" event={"ID":"0f0c847d-119c-469b-943d-e385d909b9b3","Type":"ContainerStarted","Data":"ed17ac0a2317750abdb0f9460ca2bc10a96b49676c5ec0a08b818d40287ca7be"} Jul 06 20:15:59.197607 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:59.197305 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-g9qjv" Jul 06 20:15:59.208933 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:59.208908 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-g9qjv" Jul 06 20:15:59.210878 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:59.210785 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f6dfc6666-c4s7h" podStartSLOduration=1.79414336 podStartE2EDuration="26.210770131s" totalImagesPullingTime="24.416626771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:59.210204531 +0000 UTC m=+96.056069259" watchObservedRunningTime="2026-07-06 20:15:59.210770131 +0000 UTC m=+96.056634863" Jul 06 20:15:59.230018 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:15:59.229965 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-g9qjv" podStartSLOduration=1.416312958 podStartE2EDuration="26.229947243s" totalImagesPullingTime="24.813634285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:59.228692012 +0000 UTC m=+96.074556730" watchObservedRunningTime="2026-07-06 20:15:59.229947243 +0000 UTC m=+96.075811947" Jul 06 20:16:01.000148 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:00.999225 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5slw5" Jul 06 20:16:01.208353 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:01.208269 2566 generic.go:356] "Generic (PLEG): container finished" podID="afd18897-c080-4905-98b3-17b79190b39b" containerID="fb8937bce745d38994b5a9a2aa4f2c93e96dfb71eee6f3b9271c30303a0fb913" exitCode=0 Jul 06 20:16:01.208516 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:01.208368 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2lzrs" event={"ID":"afd18897-c080-4905-98b3-17b79190b39b","Type":"ContainerDied","Data":"fb8937bce745d38994b5a9a2aa4f2c93e96dfb71eee6f3b9271c30303a0fb913"} Jul 06 20:16:01.210609 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:01.210580 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lvtdc" event={"ID":"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7","Type":"ContainerStarted","Data":"5c666bba72a8deded7bfc8f4be8ca270c9bfd3fa9de32c9a56ba19a8edcce048"} Jul 06 20:16:02.220186 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:02.220147 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2lzrs" event={"ID":"afd18897-c080-4905-98b3-17b79190b39b","Type":"ContainerStarted","Data":"d8d9bc409009287d4a40673e5c3a45aa69cef05f7bbb7f60aef71f8fe9987e16"} Jul 06 20:16:02.220842 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:02.220213 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2lzrs" event={"ID":"afd18897-c080-4905-98b3-17b79190b39b","Type":"ContainerStarted","Data":"4461dae4401f45d8a74fe87f90af2251d10e9e8fd9a214fb97437a15e72d433d"} Jul 06 20:16:02.246238 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:02.246072 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2lzrs" podStartSLOduration=6.358515935 podStartE2EDuration="8.246050053s" totalImagesPullingTime="1.785801897s" totalInitContainerRuntime="101.732221ms" isStatefulPod=true podCreationTimestamp="2026-07-06 20:15:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:02.243994964 +0000 UTC m=+99.089859682" watchObservedRunningTime="2026-07-06 20:16:02.246050053 +0000 UTC m=+99.091914759" Jul 06 20:16:05.236377 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:05.236337 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lvtdc" event={"ID":"748c7f38-4aa4-4279-b7db-b0eb53d4e9e7","Type":"ContainerStarted","Data":"b4227441ee51e1fbbd6885e0fced1951c18a73fa526a6b2971b4da5c3516f90c"} Jul 06 20:16:05.265262 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:05.265202 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lvtdc" podStartSLOduration=3.543152369 podStartE2EDuration="21.265184343s" totalImagesPullingTime="17.722031974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:15:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:05.262727729 +0000 UTC m=+102.108592434" watchObservedRunningTime="2026-07-06 20:16:05.265184343 +0000 UTC m=+102.111049048" Jul 06 20:16:22.367207 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.367163 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" podUID="a2188220-020e-4d49-95c4-a14e2aba5d35" containerName="registry" containerID="cri-o://24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534" gracePeriod=30 Jul 06 20:16:22.642070 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.642013 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:16:22.705224 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.705195 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-trusted-ca\") pod \"a2188220-020e-4d49-95c4-a14e2aba5d35\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " Jul 06 20:16:22.705224 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.705230 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2188220-020e-4d49-95c4-a14e2aba5d35-ca-trust-extracted\") pod \"a2188220-020e-4d49-95c4-a14e2aba5d35\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " Jul 06 20:16:22.705415 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.705251 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-installation-pull-secrets\") pod \"a2188220-020e-4d49-95c4-a14e2aba5d35\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " Jul 06 20:16:22.705415 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.705269 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-bound-sa-token\") pod \"a2188220-020e-4d49-95c4-a14e2aba5d35\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " Jul 06 20:16:22.705415 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.705325 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-certificates\") pod \"a2188220-020e-4d49-95c4-a14e2aba5d35\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " Jul 06 20:16:22.705415 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.705372 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-image-registry-private-configuration\") pod \"a2188220-020e-4d49-95c4-a14e2aba5d35\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " Jul 06 20:16:22.705415 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.705408 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls\") pod \"a2188220-020e-4d49-95c4-a14e2aba5d35\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " Jul 06 20:16:22.705692 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.705489 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrx2s\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-kube-api-access-hrx2s\") pod \"a2188220-020e-4d49-95c4-a14e2aba5d35\" (UID: \"a2188220-020e-4d49-95c4-a14e2aba5d35\") " Jul 06 20:16:22.705796 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.705767 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a2188220-020e-4d49-95c4-a14e2aba5d35" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:16:22.705879 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.705858 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a2188220-020e-4d49-95c4-a14e2aba5d35" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:16:22.708065 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.708034 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-kube-api-access-hrx2s" (OuterVolumeSpecName: "kube-api-access-hrx2s") pod "a2188220-020e-4d49-95c4-a14e2aba5d35" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35"). InnerVolumeSpecName "kube-api-access-hrx2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:16:22.708187 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.708099 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a2188220-020e-4d49-95c4-a14e2aba5d35" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:16:22.708187 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.708110 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a2188220-020e-4d49-95c4-a14e2aba5d35" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:16:22.708187 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.708108 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a2188220-020e-4d49-95c4-a14e2aba5d35" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:16:22.708187 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.708143 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a2188220-020e-4d49-95c4-a14e2aba5d35" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:16:22.714061 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.714033 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2188220-020e-4d49-95c4-a14e2aba5d35-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a2188220-020e-4d49-95c4-a14e2aba5d35" (UID: "a2188220-020e-4d49-95c4-a14e2aba5d35"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:16:22.806518 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.806495 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-trusted-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:16:22.806518 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.806517 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2188220-020e-4d49-95c4-a14e2aba5d35-ca-trust-extracted\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:16:22.806653 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.806526 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-installation-pull-secrets\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:16:22.806653 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.806535 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-bound-sa-token\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:16:22.806653 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.806543 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-certificates\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:16:22.806653 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.806574 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2188220-020e-4d49-95c4-a14e2aba5d35-image-registry-private-configuration\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:16:22.806653 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.806582 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-registry-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:16:22.806653 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:22.806590 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrx2s\" (UniqueName: \"kubernetes.io/projected/a2188220-020e-4d49-95c4-a14e2aba5d35-kube-api-access-hrx2s\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:16:23.307075 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:23.307038 2566 generic.go:356] "Generic (PLEG): container finished" podID="a2188220-020e-4d49-95c4-a14e2aba5d35" containerID="24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534" exitCode=0 Jul 06 20:16:23.307265 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:23.307089 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" Jul 06 20:16:23.307265 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:23.307123 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" event={"ID":"a2188220-020e-4d49-95c4-a14e2aba5d35","Type":"ContainerDied","Data":"24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534"} Jul 06 20:16:23.307265 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:23.307210 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c4ddc6c69-6q7pm" event={"ID":"a2188220-020e-4d49-95c4-a14e2aba5d35","Type":"ContainerDied","Data":"6724ab54c9cb2563333b2dd4859cef5e4171d884a07c61a99b0a92875afacc5d"} Jul 06 20:16:23.307423 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:23.307302 2566 scope.go:117] "RemoveContainer" containerID="24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534" Jul 06 20:16:23.316036 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:23.316017 2566 scope.go:117] "RemoveContainer" containerID="24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534" Jul 06 20:16:23.316301 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:16:23.316282 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534\": container with ID starting with 24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534 not found: ID does not exist" containerID="24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534" Jul 06 20:16:23.316344 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:23.316309 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534"} err="failed to get container status \"24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534\": rpc error: code = NotFound desc = could not find container \"24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534\": container with ID starting with 24dc9305ee8c7759563306d3209ca5f192165360adcb17f330cc3e5801788534 not found: ID does not exist" Jul 06 20:16:23.330306 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:23.330278 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c4ddc6c69-6q7pm"] Jul 06 20:16:23.334640 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:23.334620 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7c4ddc6c69-6q7pm"] Jul 06 20:16:23.667124 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:23.667055 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2188220-020e-4d49-95c4-a14e2aba5d35" path="/var/lib/kubelet/pods/a2188220-020e-4d49-95c4-a14e2aba5d35/volumes" Jul 06 20:16:34.349202 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:34.349166 2566 generic.go:356] "Generic (PLEG): container finished" podID="5dd21d42-e317-4776-9822-2dbb049973f2" containerID="96f23355e96d850bfa5bb073fff459d39057db3a5274c094c68949c5c9271030" exitCode=0 Jul 06 20:16:34.349615 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:34.349221 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" event={"ID":"5dd21d42-e317-4776-9822-2dbb049973f2","Type":"ContainerDied","Data":"96f23355e96d850bfa5bb073fff459d39057db3a5274c094c68949c5c9271030"} Jul 06 20:16:34.349615 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:34.349573 2566 scope.go:117] "RemoveContainer" containerID="96f23355e96d850bfa5bb073fff459d39057db3a5274c094c68949c5c9271030" Jul 06 20:16:35.354829 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:35.354800 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-w6b76" event={"ID":"5dd21d42-e317-4776-9822-2dbb049973f2","Type":"ContainerStarted","Data":"45c0c8270ca59c873749b6f5d2eccc3cff66fc891859b925bce472a2332eb479"} Jul 06 20:16:39.372561 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:39.372515 2566 generic.go:356] "Generic (PLEG): container finished" podID="5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89" containerID="6ac15cd15ec78c13c8c09fbd3f48d594722984f0d66480313accb10e89b47c62" exitCode=0 Jul 06 20:16:39.373033 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:39.372574 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" event={"ID":"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89","Type":"ContainerDied","Data":"6ac15cd15ec78c13c8c09fbd3f48d594722984f0d66480313accb10e89b47c62"} Jul 06 20:16:39.373033 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:39.372898 2566 scope.go:117] "RemoveContainer" containerID="6ac15cd15ec78c13c8c09fbd3f48d594722984f0d66480313accb10e89b47c62" Jul 06 20:16:40.377975 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:16:40.377941 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9pq4z" event={"ID":"5890eaa1-b60c-4dc1-b5df-3e3eb7da7d89","Type":"ContainerStarted","Data":"137062c2fff01a30e59383d3febcd2a5d7db01f5830c6c093d9bd8704f4bad40"} Jul 06 20:19:23.528930 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:23.528899 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:19:23.531869 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:23.531840 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:19:23.540256 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:23.540227 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 20:19:25.143066 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.143036 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q"] Jul 06 20:19:25.144798 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.143324 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2188220-020e-4d49-95c4-a14e2aba5d35" containerName="registry" Jul 06 20:19:25.144798 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.143336 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2188220-020e-4d49-95c4-a14e2aba5d35" containerName="registry" Jul 06 20:19:25.144798 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.143397 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2188220-020e-4d49-95c4-a14e2aba5d35" containerName="registry" Jul 06 20:19:25.145372 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.145356 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.148949 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.148926 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 20:19:25.149076 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.149028 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-88kxf\"" Jul 06 20:19:25.150723 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.150703 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 20:19:25.150974 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.150956 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 20:19:25.151083 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.150999 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 20:19:25.162251 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.162211 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q"] Jul 06 20:19:25.260770 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.260741 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55669313-69e2-4bb1-b40f-08e723f79c2d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.260896 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.260774 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55669313-69e2-4bb1-b40f-08e723f79c2d-manifests\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.260896 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.260797 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55669313-69e2-4bb1-b40f-08e723f79c2d-tmp\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.260896 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.260863 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj2xg\" (UniqueName: \"kubernetes.io/projected/55669313-69e2-4bb1-b40f-08e723f79c2d-kube-api-access-pj2xg\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.261013 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.260900 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55669313-69e2-4bb1-b40f-08e723f79c2d-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.361497 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.361469 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55669313-69e2-4bb1-b40f-08e723f79c2d-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.361646 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.361517 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55669313-69e2-4bb1-b40f-08e723f79c2d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.361646 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.361538 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55669313-69e2-4bb1-b40f-08e723f79c2d-manifests\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.361646 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.361577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55669313-69e2-4bb1-b40f-08e723f79c2d-tmp\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.361646 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.361612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj2xg\" (UniqueName: \"kubernetes.io/projected/55669313-69e2-4bb1-b40f-08e723f79c2d-kube-api-access-pj2xg\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.362018 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.361997 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55669313-69e2-4bb1-b40f-08e723f79c2d-manifests\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.363922 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.363895 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55669313-69e2-4bb1-b40f-08e723f79c2d-tmp\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.364023 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.363984 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55669313-69e2-4bb1-b40f-08e723f79c2d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.364084 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.364053 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55669313-69e2-4bb1-b40f-08e723f79c2d-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.369234 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.369212 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj2xg\" (UniqueName: \"kubernetes.io/projected/55669313-69e2-4bb1-b40f-08e723f79c2d-kube-api-access-pj2xg\") pod \"opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q\" (UID: \"55669313-69e2-4bb1-b40f-08e723f79c2d\") " pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.455870 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.455814 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:25.597209 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.597182 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q"] Jul 06 20:19:25.599516 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:19:25.599485 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55669313_69e2_4bb1_b40f_08e723f79c2d.slice/crio-363b0a96ec790592e458032af08e3cde0cd5e403033079529e01fe27bd2de727 WatchSource:0}: Error finding container 363b0a96ec790592e458032af08e3cde0cd5e403033079529e01fe27bd2de727: Status 404 returned error can't find the container with id 363b0a96ec790592e458032af08e3cde0cd5e403033079529e01fe27bd2de727 Jul 06 20:19:25.601301 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.601285 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:19:25.994852 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:25.994815 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" event={"ID":"55669313-69e2-4bb1-b40f-08e723f79c2d","Type":"ContainerStarted","Data":"363b0a96ec790592e458032af08e3cde0cd5e403033079529e01fe27bd2de727"} Jul 06 20:19:29.010244 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:29.010145 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" event={"ID":"55669313-69e2-4bb1-b40f-08e723f79c2d","Type":"ContainerStarted","Data":"94fd4e4d7cdb3876d8a5832c2b2466bf6f9e87cb4056fec5f0f49a2db8e2accf"} Jul 06 20:19:30.015981 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:30.015950 2566 generic.go:356] "Generic (PLEG): container finished" podID="55669313-69e2-4bb1-b40f-08e723f79c2d" containerID="94fd4e4d7cdb3876d8a5832c2b2466bf6f9e87cb4056fec5f0f49a2db8e2accf" exitCode=0 Jul 06 20:19:30.016315 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:30.016033 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" event={"ID":"55669313-69e2-4bb1-b40f-08e723f79c2d","Type":"ContainerDied","Data":"94fd4e4d7cdb3876d8a5832c2b2466bf6f9e87cb4056fec5f0f49a2db8e2accf"} Jul 06 20:19:31.022298 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:31.022264 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" event={"ID":"55669313-69e2-4bb1-b40f-08e723f79c2d","Type":"ContainerStarted","Data":"d9b74cd64d89f799db534e6542857a777c4e087fbda527a84c9d2b34a425c06c"} Jul 06 20:19:31.022702 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:31.022385 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:31.043667 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:31.043625 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" podStartSLOduration=1.504252475 podStartE2EDuration="6.043608194s" totalImagesPullingTime="3.495917768s" totalInitContainerRuntime="1.043437951s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:19:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:19:31.041505754 +0000 UTC m=+307.887370458" watchObservedRunningTime="2026-07-06 20:19:31.043608194 +0000 UTC m=+307.889472900" Jul 06 20:19:42.029828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:42.029796 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q" Jul 06 20:19:53.521212 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.521177 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg"] Jul 06 20:19:53.523690 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.523674 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.528442 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.528424 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 20:19:53.529073 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.529038 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 20:19:53.529175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.529092 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 20:19:53.529175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.529044 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-f4gw6\"" Jul 06 20:19:53.529297 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.529038 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 20:19:53.529465 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.529450 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:19:53.538443 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.538423 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg"] Jul 06 20:19:53.694339 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.694305 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msw6k\" (UniqueName: \"kubernetes.io/projected/e2a0df31-c44c-4baa-aad1-2ea07496d89a-kube-api-access-msw6k\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.694521 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.694349 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2a0df31-c44c-4baa-aad1-2ea07496d89a-cert\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.694521 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.694402 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e2a0df31-c44c-4baa-aad1-2ea07496d89a-manager-config\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.694521 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.694428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2a0df31-c44c-4baa-aad1-2ea07496d89a-metrics-cert\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.795375 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.795296 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e2a0df31-c44c-4baa-aad1-2ea07496d89a-manager-config\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.795375 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.795331 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2a0df31-c44c-4baa-aad1-2ea07496d89a-metrics-cert\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.795631 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.795390 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msw6k\" (UniqueName: \"kubernetes.io/projected/e2a0df31-c44c-4baa-aad1-2ea07496d89a-kube-api-access-msw6k\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.795631 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.795423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2a0df31-c44c-4baa-aad1-2ea07496d89a-cert\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.795965 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.795945 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e2a0df31-c44c-4baa-aad1-2ea07496d89a-manager-config\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.797916 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.797893 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2a0df31-c44c-4baa-aad1-2ea07496d89a-metrics-cert\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.798068 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.798045 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2a0df31-c44c-4baa-aad1-2ea07496d89a-cert\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.810925 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.810899 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msw6k\" (UniqueName: \"kubernetes.io/projected/e2a0df31-c44c-4baa-aad1-2ea07496d89a-kube-api-access-msw6k\") pod \"lws-controller-manager-86b486bf74-r8brg\" (UID: \"e2a0df31-c44c-4baa-aad1-2ea07496d89a\") " pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.832746 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.832720 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:53.991681 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:53.991657 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg"] Jul 06 20:19:53.994008 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:19:53.993980 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2a0df31_c44c_4baa_aad1_2ea07496d89a.slice/crio-6e9d4df87975417d3c43de09008fb1c3bfe509e3ab36ccb6bd30e3fabd82fcba WatchSource:0}: Error finding container 6e9d4df87975417d3c43de09008fb1c3bfe509e3ab36ccb6bd30e3fabd82fcba: Status 404 returned error can't find the container with id 6e9d4df87975417d3c43de09008fb1c3bfe509e3ab36ccb6bd30e3fabd82fcba Jul 06 20:19:54.123338 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:54.123256 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" event={"ID":"e2a0df31-c44c-4baa-aad1-2ea07496d89a","Type":"ContainerStarted","Data":"6e9d4df87975417d3c43de09008fb1c3bfe509e3ab36ccb6bd30e3fabd82fcba"} Jul 06 20:19:57.140259 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:57.140219 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" event={"ID":"e2a0df31-c44c-4baa-aad1-2ea07496d89a","Type":"ContainerStarted","Data":"da25098348224511787d775d7530a5ed790d150b8b6976cae8b2a01ea0f65175"} Jul 06 20:19:57.140706 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:57.140323 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:19:57.169108 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:19:57.169062 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" podStartSLOduration=1.669808943 podStartE2EDuration="4.169047479s" totalImagesPullingTime="2.499238536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:19:57.16511687 +0000 UTC m=+334.010981574" watchObservedRunningTime="2026-07-06 20:19:57.169047479 +0000 UTC m=+334.014912184" Jul 06 20:20:08.149205 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:20:08.149171 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-86b486bf74-r8brg" Jul 06 20:21:24.451131 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.451096 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg"] Jul 06 20:21:24.453499 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.453477 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg" Jul 06 20:21:24.458069 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.458046 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 20:21:24.458260 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.458233 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-76jk5\"" Jul 06 20:21:24.459373 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.459350 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 20:21:24.459495 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.459381 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 20:21:24.472404 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.472379 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg"] Jul 06 20:21:24.545085 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.545047 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj7cl\" (UniqueName: \"kubernetes.io/projected/76fac1fe-66ba-4da1-a360-a2420fd87e35-kube-api-access-hj7cl\") pod \"dns-operator-controller-manager-864dd78c4c-rxdxg\" (UID: \"76fac1fe-66ba-4da1-a360-a2420fd87e35\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg" Jul 06 20:21:24.646345 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.646312 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj7cl\" (UniqueName: \"kubernetes.io/projected/76fac1fe-66ba-4da1-a360-a2420fd87e35-kube-api-access-hj7cl\") pod \"dns-operator-controller-manager-864dd78c4c-rxdxg\" (UID: \"76fac1fe-66ba-4da1-a360-a2420fd87e35\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg" Jul 06 20:21:24.655160 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.655131 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj7cl\" (UniqueName: \"kubernetes.io/projected/76fac1fe-66ba-4da1-a360-a2420fd87e35-kube-api-access-hj7cl\") pod \"dns-operator-controller-manager-864dd78c4c-rxdxg\" (UID: \"76fac1fe-66ba-4da1-a360-a2420fd87e35\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg" Jul 06 20:21:24.766678 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.766644 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg" Jul 06 20:21:24.915872 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:24.915828 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg"] Jul 06 20:21:24.920890 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:21:24.920866 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76fac1fe_66ba_4da1_a360_a2420fd87e35.slice/crio-29510bf0552ccc74da8d766242f54698dd1f451f764d98bfadda05361e20ed37 WatchSource:0}: Error finding container 29510bf0552ccc74da8d766242f54698dd1f451f764d98bfadda05361e20ed37: Status 404 returned error can't find the container with id 29510bf0552ccc74da8d766242f54698dd1f451f764d98bfadda05361e20ed37 Jul 06 20:21:25.509210 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:25.509171 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg" event={"ID":"76fac1fe-66ba-4da1-a360-a2420fd87e35","Type":"ContainerStarted","Data":"29510bf0552ccc74da8d766242f54698dd1f451f764d98bfadda05361e20ed37"} Jul 06 20:21:26.450175 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:26.450140 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b"] Jul 06 20:21:26.453393 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:26.453373 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:26.456275 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:26.456256 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-r5zlg\"" Jul 06 20:21:26.477810 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:26.477785 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b"] Jul 06 20:21:26.562767 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:26.562733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmfs8\" (UniqueName: \"kubernetes.io/projected/c74b9e97-d973-40b9-a39c-d8b7ff3a8739-kube-api-access-nmfs8\") pod \"limitador-operator-controller-manager-859d7b78bd-ds54b\" (UID: \"c74b9e97-d973-40b9-a39c-d8b7ff3a8739\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:26.663564 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:26.663514 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmfs8\" (UniqueName: \"kubernetes.io/projected/c74b9e97-d973-40b9-a39c-d8b7ff3a8739-kube-api-access-nmfs8\") pod \"limitador-operator-controller-manager-859d7b78bd-ds54b\" (UID: \"c74b9e97-d973-40b9-a39c-d8b7ff3a8739\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:26.681180 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:26.681139 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmfs8\" (UniqueName: \"kubernetes.io/projected/c74b9e97-d973-40b9-a39c-d8b7ff3a8739-kube-api-access-nmfs8\") pod \"limitador-operator-controller-manager-859d7b78bd-ds54b\" (UID: \"c74b9e97-d973-40b9-a39c-d8b7ff3a8739\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:26.767292 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:26.767259 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:27.144021 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:27.143992 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b"] Jul 06 20:21:27.146675 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:21:27.146651 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc74b9e97_d973_40b9_a39c_d8b7ff3a8739.slice/crio-c1dda505be01bad4d8d24391c262e81167be5ffa40ce3c441fad90efcdbee393 WatchSource:0}: Error finding container c1dda505be01bad4d8d24391c262e81167be5ffa40ce3c441fad90efcdbee393: Status 404 returned error can't find the container with id c1dda505be01bad4d8d24391c262e81167be5ffa40ce3c441fad90efcdbee393 Jul 06 20:21:27.518656 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:27.518615 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg" event={"ID":"76fac1fe-66ba-4da1-a360-a2420fd87e35","Type":"ContainerStarted","Data":"d758247ef67b41afaa4bfc77e4368d5f5dca72ba2bc734bf18bf3c495d8d4c23"} Jul 06 20:21:27.518830 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:27.518694 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg" Jul 06 20:21:27.519931 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:27.519904 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" event={"ID":"c74b9e97-d973-40b9-a39c-d8b7ff3a8739","Type":"ContainerStarted","Data":"c1dda505be01bad4d8d24391c262e81167be5ffa40ce3c441fad90efcdbee393"} Jul 06 20:21:27.565914 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:27.565863 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg" podStartSLOduration=1.4236441 podStartE2EDuration="3.565843347s" totalImagesPullingTime="2.142199247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:27.563525594 +0000 UTC m=+424.409390299" watchObservedRunningTime="2026-07-06 20:21:27.565843347 +0000 UTC m=+424.411708052" Jul 06 20:21:29.531785 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:29.531696 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" event={"ID":"c74b9e97-d973-40b9-a39c-d8b7ff3a8739","Type":"ContainerStarted","Data":"b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019"} Jul 06 20:21:29.532167 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:29.531819 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:29.551214 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:29.551170 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" podStartSLOduration=1.4320953570000001 podStartE2EDuration="3.551157536s" totalImagesPullingTime="2.119062179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:29.548946646 +0000 UTC m=+426.394811360" watchObservedRunningTime="2026-07-06 20:21:29.551157536 +0000 UTC m=+426.397022286" Jul 06 20:21:38.527963 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:38.527933 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rxdxg" Jul 06 20:21:40.059922 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.059889 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b"] Jul 06 20:21:40.060338 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.060087 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" podUID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" containerName="manager" containerID="cri-o://b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019" gracePeriod=2 Jul 06 20:21:40.061918 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.061886 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:40.067047 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.067021 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b"] Jul 06 20:21:40.081289 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:21:40.081258 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ds54b\" is forbidden: User \"system:node:ip-10-0-133-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-106.ec2.internal' and this object" podUID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:40.129186 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.129156 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb"] Jul 06 20:21:40.129475 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.129463 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" containerName="manager" Jul 06 20:21:40.129523 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.129477 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" containerName="manager" Jul 06 20:21:40.129583 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.129570 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" containerName="manager" Jul 06 20:21:40.132586 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.132565 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb" Jul 06 20:21:40.178522 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:21:40.178492 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ds54b\" is forbidden: User \"system:node:ip-10-0-133-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-106.ec2.internal' and this object" podUID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:40.226560 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.226523 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb"] Jul 06 20:21:40.279366 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.279331 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9b2x\" (UniqueName: \"kubernetes.io/projected/cfd55d03-32f0-4c13-833f-bf53e029ce80-kube-api-access-n9b2x\") pod \"limitador-operator-controller-manager-859d7b78bd-74nrb\" (UID: \"cfd55d03-32f0-4c13-833f-bf53e029ce80\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb" Jul 06 20:21:40.327961 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.327936 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:40.331442 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:21:40.331420 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ds54b\" is forbidden: User \"system:node:ip-10-0-133-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-106.ec2.internal' and this object" podUID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:40.379987 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.379954 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9b2x\" (UniqueName: \"kubernetes.io/projected/cfd55d03-32f0-4c13-833f-bf53e029ce80-kube-api-access-n9b2x\") pod \"limitador-operator-controller-manager-859d7b78bd-74nrb\" (UID: \"cfd55d03-32f0-4c13-833f-bf53e029ce80\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb" Jul 06 20:21:40.396476 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.396448 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9b2x\" (UniqueName: \"kubernetes.io/projected/cfd55d03-32f0-4c13-833f-bf53e029ce80-kube-api-access-n9b2x\") pod \"limitador-operator-controller-manager-859d7b78bd-74nrb\" (UID: \"cfd55d03-32f0-4c13-833f-bf53e029ce80\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb" Jul 06 20:21:40.446049 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.446009 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb" Jul 06 20:21:40.481108 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.481070 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmfs8\" (UniqueName: \"kubernetes.io/projected/c74b9e97-d973-40b9-a39c-d8b7ff3a8739-kube-api-access-nmfs8\") pod \"c74b9e97-d973-40b9-a39c-d8b7ff3a8739\" (UID: \"c74b9e97-d973-40b9-a39c-d8b7ff3a8739\") " Jul 06 20:21:40.483129 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.483101 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c74b9e97-d973-40b9-a39c-d8b7ff3a8739-kube-api-access-nmfs8" (OuterVolumeSpecName: "kube-api-access-nmfs8") pod "c74b9e97-d973-40b9-a39c-d8b7ff3a8739" (UID: "c74b9e97-d973-40b9-a39c-d8b7ff3a8739"). InnerVolumeSpecName "kube-api-access-nmfs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:21:40.580635 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.580532 2566 generic.go:356] "Generic (PLEG): container finished" podID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" containerID="b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019" exitCode=0 Jul 06 20:21:40.580635 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.580598 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:40.580635 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.580605 2566 scope.go:117] "RemoveContainer" containerID="b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019" Jul 06 20:21:40.581724 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.581676 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmfs8\" (UniqueName: \"kubernetes.io/projected/c74b9e97-d973-40b9-a39c-d8b7ff3a8739-kube-api-access-nmfs8\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:21:40.583221 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:21:40.583191 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ds54b\" is forbidden: User \"system:node:ip-10-0-133-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-106.ec2.internal' and this object" podUID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:40.587210 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.587191 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb"] Jul 06 20:21:40.590213 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.590197 2566 scope.go:117] "RemoveContainer" containerID="b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019" Jul 06 20:21:40.590482 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:21:40.590463 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019\": container with ID starting with b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019 not found: ID does not exist" containerID="b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019" Jul 06 20:21:40.590582 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:40.590488 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019"} err="failed to get container status \"b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019\": rpc error: code = NotFound desc = could not find container \"b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019\": container with ID starting with b33f40e2e6549d9bd091b532764bd1cdac9159f7744b8964cad1181b48f0d019 not found: ID does not exist" Jul 06 20:21:40.591662 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:21:40.591645 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd55d03_32f0_4c13_833f_bf53e029ce80.slice/crio-1bf3a091168acc46acfbcc41d9943c2d1a66867d73c4e4108385826b25120c66 WatchSource:0}: Error finding container 1bf3a091168acc46acfbcc41d9943c2d1a66867d73c4e4108385826b25120c66: Status 404 returned error can't find the container with id 1bf3a091168acc46acfbcc41d9943c2d1a66867d73c4e4108385826b25120c66 Jul 06 20:21:40.593071 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:21:40.593044 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ds54b\" is forbidden: User \"system:node:ip-10-0-133-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-106.ec2.internal' and this object" podUID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:41.587495 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:41.587460 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb" event={"ID":"cfd55d03-32f0-4c13-833f-bf53e029ce80","Type":"ContainerStarted","Data":"93c5445c8b74723c074a156d43f5e5aed034a7792a5add472718df785b53985b"} Jul 06 20:21:41.587858 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:41.587512 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb" event={"ID":"cfd55d03-32f0-4c13-833f-bf53e029ce80","Type":"ContainerStarted","Data":"1bf3a091168acc46acfbcc41d9943c2d1a66867d73c4e4108385826b25120c66"} Jul 06 20:21:41.587858 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:41.587535 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb" Jul 06 20:21:41.621207 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:41.621166 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb" podStartSLOduration=1.621152223 podStartE2EDuration="1.621152223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:41.620724943 +0000 UTC m=+438.466589648" watchObservedRunningTime="2026-07-06 20:21:41.621152223 +0000 UTC m=+438.467016930" Jul 06 20:21:41.622823 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:21:41.622799 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ds54b\" is forbidden: User \"system:node:ip-10-0-133-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-106.ec2.internal' and this object" podUID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ds54b" Jul 06 20:21:41.667899 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:41.667864 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c74b9e97-d973-40b9-a39c-d8b7ff3a8739" path="/var/lib/kubelet/pods/c74b9e97-d973-40b9-a39c-d8b7ff3a8739/volumes" Jul 06 20:21:52.594581 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:21:52.594482 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-74nrb" Jul 06 20:22:25.783269 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.783238 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:22:25.787857 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.787831 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" Jul 06 20:22:25.790288 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.790268 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 20:22:25.790391 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.790309 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-c29nn\"" Jul 06 20:22:25.797265 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.797242 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:22:25.812645 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.812621 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:22:25.845094 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.845069 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n66gg\" (UniqueName: \"kubernetes.io/projected/9c1888d6-b8b6-489d-a838-1971f31bb10d-kube-api-access-n66gg\") pod \"limitador-limitador-54d6d988d5-fhvmh\" (UID: \"9c1888d6-b8b6-489d-a838-1971f31bb10d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" Jul 06 20:22:25.845205 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.845109 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c1888d6-b8b6-489d-a838-1971f31bb10d-config-file\") pod \"limitador-limitador-54d6d988d5-fhvmh\" (UID: \"9c1888d6-b8b6-489d-a838-1971f31bb10d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" Jul 06 20:22:25.945909 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.945881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n66gg\" (UniqueName: \"kubernetes.io/projected/9c1888d6-b8b6-489d-a838-1971f31bb10d-kube-api-access-n66gg\") pod \"limitador-limitador-54d6d988d5-fhvmh\" (UID: \"9c1888d6-b8b6-489d-a838-1971f31bb10d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" Jul 06 20:22:25.946034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.945934 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c1888d6-b8b6-489d-a838-1971f31bb10d-config-file\") pod \"limitador-limitador-54d6d988d5-fhvmh\" (UID: \"9c1888d6-b8b6-489d-a838-1971f31bb10d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" Jul 06 20:22:25.946571 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.946530 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9c1888d6-b8b6-489d-a838-1971f31bb10d-config-file\") pod \"limitador-limitador-54d6d988d5-fhvmh\" (UID: \"9c1888d6-b8b6-489d-a838-1971f31bb10d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" Jul 06 20:22:25.955804 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:25.955782 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n66gg\" (UniqueName: \"kubernetes.io/projected/9c1888d6-b8b6-489d-a838-1971f31bb10d-kube-api-access-n66gg\") pod \"limitador-limitador-54d6d988d5-fhvmh\" (UID: \"9c1888d6-b8b6-489d-a838-1971f31bb10d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" Jul 06 20:22:26.098931 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:26.098861 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" Jul 06 20:22:26.241646 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:26.241609 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:22:26.244859 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:22:26.244833 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c1888d6_b8b6_489d_a838_1971f31bb10d.slice/crio-91f86257b626964deb4a32e04417fb524bb0f19c45c3cc9a0507bea3f49433e4 WatchSource:0}: Error finding container 91f86257b626964deb4a32e04417fb524bb0f19c45c3cc9a0507bea3f49433e4: Status 404 returned error can't find the container with id 91f86257b626964deb4a32e04417fb524bb0f19c45c3cc9a0507bea3f49433e4 Jul 06 20:22:26.797696 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:26.797657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" event={"ID":"9c1888d6-b8b6-489d-a838-1971f31bb10d","Type":"ContainerStarted","Data":"91f86257b626964deb4a32e04417fb524bb0f19c45c3cc9a0507bea3f49433e4"} Jul 06 20:22:28.810773 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:28.810735 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" event={"ID":"9c1888d6-b8b6-489d-a838-1971f31bb10d","Type":"ContainerStarted","Data":"0aa807307526660a505213939e905fc5410f7ebed40992770429f9a95a903426"} Jul 06 20:22:28.811177 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:28.810888 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" Jul 06 20:22:28.836873 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:28.836819 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" podStartSLOduration=1.409174819 podStartE2EDuration="3.836805249s" totalImagesPullingTime="2.42763043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:22:28.836119968 +0000 UTC m=+485.681984685" watchObservedRunningTime="2026-07-06 20:22:28.836805249 +0000 UTC m=+485.682669954" Jul 06 20:22:39.816379 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:39.816350 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-fhvmh" Jul 06 20:22:54.249756 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:54.249717 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6bbf94ffd9-qj9g2"] Jul 06 20:22:54.252800 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:54.252776 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" Jul 06 20:22:54.255433 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:54.255417 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2m962\"" Jul 06 20:22:54.260370 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:54.260346 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bbf94ffd9-qj9g2"] Jul 06 20:22:54.387304 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:54.387268 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24tkp\" (UniqueName: \"kubernetes.io/projected/396b1515-7a1d-4039-81ab-edfcc38d0b20-kube-api-access-24tkp\") pod \"authorino-6bbf94ffd9-qj9g2\" (UID: \"396b1515-7a1d-4039-81ab-edfcc38d0b20\") " pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" Jul 06 20:22:54.488096 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:54.488063 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24tkp\" (UniqueName: \"kubernetes.io/projected/396b1515-7a1d-4039-81ab-edfcc38d0b20-kube-api-access-24tkp\") pod \"authorino-6bbf94ffd9-qj9g2\" (UID: \"396b1515-7a1d-4039-81ab-edfcc38d0b20\") " pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" Jul 06 20:22:54.496620 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:54.496589 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24tkp\" (UniqueName: \"kubernetes.io/projected/396b1515-7a1d-4039-81ab-edfcc38d0b20-kube-api-access-24tkp\") pod \"authorino-6bbf94ffd9-qj9g2\" (UID: \"396b1515-7a1d-4039-81ab-edfcc38d0b20\") " pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" Jul 06 20:22:54.562394 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:54.562324 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" Jul 06 20:22:54.579025 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:54.579001 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bbf94ffd9-qj9g2"] Jul 06 20:22:54.690907 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:54.690812 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bbf94ffd9-qj9g2"] Jul 06 20:22:54.693592 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:22:54.693540 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod396b1515_7a1d_4039_81ab_edfcc38d0b20.slice/crio-d6fa4e595129862f15ead9f0428300a7ee5d3936c9e7d02c946ce0e59f4fa36f WatchSource:0}: Error finding container d6fa4e595129862f15ead9f0428300a7ee5d3936c9e7d02c946ce0e59f4fa36f: Status 404 returned error can't find the container with id d6fa4e595129862f15ead9f0428300a7ee5d3936c9e7d02c946ce0e59f4fa36f Jul 06 20:22:54.926110 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:54.926033 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" event={"ID":"396b1515-7a1d-4039-81ab-edfcc38d0b20","Type":"ContainerStarted","Data":"d6fa4e595129862f15ead9f0428300a7ee5d3936c9e7d02c946ce0e59f4fa36f"} Jul 06 20:22:58.946671 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:58.946632 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" event={"ID":"396b1515-7a1d-4039-81ab-edfcc38d0b20","Type":"ContainerStarted","Data":"c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5"} Jul 06 20:22:58.947095 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:58.946685 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" podUID="396b1515-7a1d-4039-81ab-edfcc38d0b20" containerName="authorino" containerID="cri-o://c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5" gracePeriod=30 Jul 06 20:22:58.960828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:58.960784 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" podStartSLOduration=1.702281099 podStartE2EDuration="4.960770723s" totalImagesPullingTime="3.258489624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:22:58.959828471 +0000 UTC m=+515.805693175" watchObservedRunningTime="2026-07-06 20:22:58.960770723 +0000 UTC m=+515.806635428" Jul 06 20:22:59.196200 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.196174 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" Jul 06 20:22:59.223275 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.223225 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24tkp\" (UniqueName: \"kubernetes.io/projected/396b1515-7a1d-4039-81ab-edfcc38d0b20-kube-api-access-24tkp\") pod \"396b1515-7a1d-4039-81ab-edfcc38d0b20\" (UID: \"396b1515-7a1d-4039-81ab-edfcc38d0b20\") " Jul 06 20:22:59.225181 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.225161 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/396b1515-7a1d-4039-81ab-edfcc38d0b20-kube-api-access-24tkp" (OuterVolumeSpecName: "kube-api-access-24tkp") pod "396b1515-7a1d-4039-81ab-edfcc38d0b20" (UID: "396b1515-7a1d-4039-81ab-edfcc38d0b20"). InnerVolumeSpecName "kube-api-access-24tkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:22:59.324447 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.324424 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24tkp\" (UniqueName: \"kubernetes.io/projected/396b1515-7a1d-4039-81ab-edfcc38d0b20-kube-api-access-24tkp\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:22:59.951746 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.951674 2566 generic.go:356] "Generic (PLEG): container finished" podID="396b1515-7a1d-4039-81ab-edfcc38d0b20" containerID="c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5" exitCode=0 Jul 06 20:22:59.951746 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.951723 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" Jul 06 20:22:59.952191 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.951761 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" event={"ID":"396b1515-7a1d-4039-81ab-edfcc38d0b20","Type":"ContainerDied","Data":"c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5"} Jul 06 20:22:59.952191 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.951807 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bbf94ffd9-qj9g2" event={"ID":"396b1515-7a1d-4039-81ab-edfcc38d0b20","Type":"ContainerDied","Data":"d6fa4e595129862f15ead9f0428300a7ee5d3936c9e7d02c946ce0e59f4fa36f"} Jul 06 20:22:59.952191 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.951831 2566 scope.go:117] "RemoveContainer" containerID="c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5" Jul 06 20:22:59.960039 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.960020 2566 scope.go:117] "RemoveContainer" containerID="c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5" Jul 06 20:22:59.960274 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:22:59.960258 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5\": container with ID starting with c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5 not found: ID does not exist" containerID="c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5" Jul 06 20:22:59.960321 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.960282 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5"} err="failed to get container status \"c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5\": rpc error: code = NotFound desc = could not find container \"c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5\": container with ID starting with c1b1ba0e4994435bb8d56918504d47593a533762fa7cf9c026d27a47a6492aa5 not found: ID does not exist" Jul 06 20:22:59.967749 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.967722 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bbf94ffd9-qj9g2"] Jul 06 20:22:59.972496 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:22:59.972477 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6bbf94ffd9-qj9g2"] Jul 06 20:23:01.668459 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:01.668424 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="396b1515-7a1d-4039-81ab-edfcc38d0b20" path="/var/lib/kubelet/pods/396b1515-7a1d-4039-81ab-edfcc38d0b20/volumes" Jul 06 20:23:37.621680 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.621644 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-c24v6"] Jul 06 20:23:37.622266 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.622178 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="396b1515-7a1d-4039-81ab-edfcc38d0b20" containerName="authorino" Jul 06 20:23:37.622266 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.622199 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="396b1515-7a1d-4039-81ab-edfcc38d0b20" containerName="authorino" Jul 06 20:23:37.622405 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.622297 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="396b1515-7a1d-4039-81ab-edfcc38d0b20" containerName="authorino" Jul 06 20:23:37.626874 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.626852 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" Jul 06 20:23:37.629121 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.629099 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5b5n5\"" Jul 06 20:23:37.629241 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.629106 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 20:23:37.636531 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.636508 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-c24v6"] Jul 06 20:23:37.646128 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.646109 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5l9k8"] Jul 06 20:23:37.648728 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.648687 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" Jul 06 20:23:37.660616 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.660594 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5l9k8"] Jul 06 20:23:37.734010 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.733981 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:23:37.758730 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.758698 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8p5c\" (UniqueName: \"kubernetes.io/projected/e4212cd5-2741-45bf-a2bc-e25398deca6b-kube-api-access-x8p5c\") pod \"payload-processing-58f86c7798-5l9k8\" (UID: \"e4212cd5-2741-45bf-a2bc-e25398deca6b\") " pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" Jul 06 20:23:37.758852 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.758742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brvcx\" (UniqueName: \"kubernetes.io/projected/519cc82d-24cb-4f24-944d-23e2cbed9baf-kube-api-access-brvcx\") pod \"payload-pre-processing-7964457696-c24v6\" (UID: \"519cc82d-24cb-4f24-944d-23e2cbed9baf\") " pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" Jul 06 20:23:37.758852 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.758789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/519cc82d-24cb-4f24-944d-23e2cbed9baf-plugins-config-volume\") pod \"payload-pre-processing-7964457696-c24v6\" (UID: \"519cc82d-24cb-4f24-944d-23e2cbed9baf\") " pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" Jul 06 20:23:37.758852 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.758821 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4212cd5-2741-45bf-a2bc-e25398deca6b-plugins-config-volume\") pod \"payload-processing-58f86c7798-5l9k8\" (UID: \"e4212cd5-2741-45bf-a2bc-e25398deca6b\") " pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" Jul 06 20:23:37.859277 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.859241 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/519cc82d-24cb-4f24-944d-23e2cbed9baf-plugins-config-volume\") pod \"payload-pre-processing-7964457696-c24v6\" (UID: \"519cc82d-24cb-4f24-944d-23e2cbed9baf\") " pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" Jul 06 20:23:37.859462 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.859304 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4212cd5-2741-45bf-a2bc-e25398deca6b-plugins-config-volume\") pod \"payload-processing-58f86c7798-5l9k8\" (UID: \"e4212cd5-2741-45bf-a2bc-e25398deca6b\") " pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" Jul 06 20:23:37.859462 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.859365 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8p5c\" (UniqueName: \"kubernetes.io/projected/e4212cd5-2741-45bf-a2bc-e25398deca6b-kube-api-access-x8p5c\") pod \"payload-processing-58f86c7798-5l9k8\" (UID: \"e4212cd5-2741-45bf-a2bc-e25398deca6b\") " pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" Jul 06 20:23:37.859462 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.859394 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brvcx\" (UniqueName: \"kubernetes.io/projected/519cc82d-24cb-4f24-944d-23e2cbed9baf-kube-api-access-brvcx\") pod \"payload-pre-processing-7964457696-c24v6\" (UID: \"519cc82d-24cb-4f24-944d-23e2cbed9baf\") " pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" Jul 06 20:23:37.860089 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.860062 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/519cc82d-24cb-4f24-944d-23e2cbed9baf-plugins-config-volume\") pod \"payload-pre-processing-7964457696-c24v6\" (UID: \"519cc82d-24cb-4f24-944d-23e2cbed9baf\") " pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" Jul 06 20:23:37.860181 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.860154 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4212cd5-2741-45bf-a2bc-e25398deca6b-plugins-config-volume\") pod \"payload-processing-58f86c7798-5l9k8\" (UID: \"e4212cd5-2741-45bf-a2bc-e25398deca6b\") " pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" Jul 06 20:23:37.868275 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.868251 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brvcx\" (UniqueName: \"kubernetes.io/projected/519cc82d-24cb-4f24-944d-23e2cbed9baf-kube-api-access-brvcx\") pod \"payload-pre-processing-7964457696-c24v6\" (UID: \"519cc82d-24cb-4f24-944d-23e2cbed9baf\") " pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" Jul 06 20:23:37.868355 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.868295 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8p5c\" (UniqueName: \"kubernetes.io/projected/e4212cd5-2741-45bf-a2bc-e25398deca6b-kube-api-access-x8p5c\") pod \"payload-processing-58f86c7798-5l9k8\" (UID: \"e4212cd5-2741-45bf-a2bc-e25398deca6b\") " pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" Jul 06 20:23:37.937475 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.937411 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" Jul 06 20:23:37.958322 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:37.958298 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" Jul 06 20:23:38.089178 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:38.089154 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-c24v6"] Jul 06 20:23:38.092600 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:23:38.092518 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod519cc82d_24cb_4f24_944d_23e2cbed9baf.slice/crio-a68d5f4f81e851e624274d6df5850a628556ec5909ddb44a051ba06240d5edaf WatchSource:0}: Error finding container a68d5f4f81e851e624274d6df5850a628556ec5909ddb44a051ba06240d5edaf: Status 404 returned error can't find the container with id a68d5f4f81e851e624274d6df5850a628556ec5909ddb44a051ba06240d5edaf Jul 06 20:23:38.112409 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:38.112382 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5l9k8"] Jul 06 20:23:38.114934 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:23:38.114911 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4212cd5_2741_45bf_a2bc_e25398deca6b.slice/crio-1292c472e4e5daeb7d57e70aeca8e8b901dd35ec1dbe0580e59576ea49662abd WatchSource:0}: Error finding container 1292c472e4e5daeb7d57e70aeca8e8b901dd35ec1dbe0580e59576ea49662abd: Status 404 returned error can't find the container with id 1292c472e4e5daeb7d57e70aeca8e8b901dd35ec1dbe0580e59576ea49662abd Jul 06 20:23:38.119665 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:38.119643 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" event={"ID":"519cc82d-24cb-4f24-944d-23e2cbed9baf","Type":"ContainerStarted","Data":"a68d5f4f81e851e624274d6df5850a628556ec5909ddb44a051ba06240d5edaf"} Jul 06 20:23:39.125035 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:39.125001 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" event={"ID":"e4212cd5-2741-45bf-a2bc-e25398deca6b","Type":"ContainerStarted","Data":"1292c472e4e5daeb7d57e70aeca8e8b901dd35ec1dbe0580e59576ea49662abd"} Jul 06 20:23:40.130632 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:40.130596 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" event={"ID":"e4212cd5-2741-45bf-a2bc-e25398deca6b","Type":"ContainerStarted","Data":"d378f7adcd6b143dcdc827b9d88e2c5706bbf53773fe934f2e145441cbb5f81e"} Jul 06 20:23:40.131113 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:40.130787 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" Jul 06 20:23:40.132092 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:40.132067 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" event={"ID":"519cc82d-24cb-4f24-944d-23e2cbed9baf","Type":"ContainerStarted","Data":"8bd103b2992a2e977f7dec9333fc2b09422950643a757ce7734ab5ca807620e5"} Jul 06 20:23:40.132196 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:40.132127 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" Jul 06 20:23:40.159900 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:40.159849 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" podStartSLOduration=1.756652235 podStartE2EDuration="3.159831201s" totalImagesPullingTime="1.403178966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:23:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:40.15805169 +0000 UTC m=+557.003916396" watchObservedRunningTime="2026-07-06 20:23:40.159831201 +0000 UTC m=+557.005695899" Jul 06 20:23:40.184018 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:40.183974 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" podStartSLOduration=1.763895064 podStartE2EDuration="3.183961624s" totalImagesPullingTime="1.42006656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:23:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:40.182811651 +0000 UTC m=+557.028676359" watchObservedRunningTime="2026-07-06 20:23:40.183961624 +0000 UTC m=+557.029826328" Jul 06 20:23:51.138649 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:51.138620 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-c24v6" Jul 06 20:23:51.139027 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:23:51.138689 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-5l9k8" Jul 06 20:24:01.273577 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:01.273527 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:24:17.748007 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.747967 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn"] Jul 06 20:24:17.750404 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.750383 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.753617 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.753591 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 20:24:17.753741 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.753726 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sf5s6\"" Jul 06 20:24:17.754795 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.754773 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 20:24:17.754897 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.754773 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 20:24:17.762940 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.762917 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn"] Jul 06 20:24:17.889830 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.889791 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.890050 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.889847 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.890050 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.889932 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.890050 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.889980 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.890050 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.890011 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.890220 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.890107 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkgpm\" (UniqueName: \"kubernetes.io/projected/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-kube-api-access-dkgpm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.990615 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.990575 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.990615 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.990620 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.990928 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.990643 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.990928 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.990708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkgpm\" (UniqueName: \"kubernetes.io/projected/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-kube-api-access-dkgpm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.990928 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.990747 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.990928 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.990793 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.991130 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.991043 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.991181 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.991124 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.991276 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.991253 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.993004 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.992985 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.993173 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.993153 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:17.999799 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:17.999738 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkgpm\" (UniqueName: \"kubernetes.io/projected/12288c3e-b505-4ef0-aa22-14b9d0f00f5b-kube-api-access-dkgpm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqmmn\" (UID: \"12288c3e-b505-4ef0-aa22-14b9d0f00f5b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:18.060919 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:18.060879 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:18.197084 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:18.197052 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn"] Jul 06 20:24:18.200613 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:24:18.200583 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12288c3e_b505_4ef0_aa22_14b9d0f00f5b.slice/crio-48d279c21621d9830745ef5b5bb17daebe03f90d76b08b56a7d894f07ce3d4fc WatchSource:0}: Error finding container 48d279c21621d9830745ef5b5bb17daebe03f90d76b08b56a7d894f07ce3d4fc: Status 404 returned error can't find the container with id 48d279c21621d9830745ef5b5bb17daebe03f90d76b08b56a7d894f07ce3d4fc Jul 06 20:24:18.305643 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:18.305543 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" event={"ID":"12288c3e-b505-4ef0-aa22-14b9d0f00f5b","Type":"ContainerStarted","Data":"48d279c21621d9830745ef5b5bb17daebe03f90d76b08b56a7d894f07ce3d4fc"} Jul 06 20:24:18.543561 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:18.543521 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:24:20.316236 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:20.316176 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" event={"ID":"12288c3e-b505-4ef0-aa22-14b9d0f00f5b","Type":"ContainerStarted","Data":"310081d739371141db58bd6de387589818272d20ac75084ef2eccde85bdd03d0"} Jul 06 20:24:23.559539 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:23.559508 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:24:23.567775 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:23.567752 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:24:28.061244 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:28.061210 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:28.062975 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:28.061272 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:28.074089 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:28.074065 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:28.091442 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:28.091395 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" podStartSLOduration=10.040289062 podStartE2EDuration="11.091380345s" totalImagesPullingTime="1.051091283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:24:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:20.336786695 +0000 UTC m=+597.182651400" watchObservedRunningTime="2026-07-06 20:24:28.091380345 +0000 UTC m=+604.937245050" Jul 06 20:24:28.374254 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:28.374166 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqmmn" Jul 06 20:24:32.340219 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:32.340185 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:24:44.330243 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:44.330205 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:24:47.264844 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.264754 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r"] Jul 06 20:24:47.267734 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.267705 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.272846 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.272796 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 20:24:47.284095 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.284073 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r"] Jul 06 20:24:47.328215 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.328183 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.328363 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.328233 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.328363 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.328275 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.328363 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.328314 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzprz\" (UniqueName: \"kubernetes.io/projected/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-kube-api-access-pzprz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.328506 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.328380 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.328506 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.328407 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.428792 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.428742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.428960 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.428807 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.428960 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.428833 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.428960 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.428869 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzprz\" (UniqueName: \"kubernetes.io/projected/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-kube-api-access-pzprz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.428960 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.428932 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.428960 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.428955 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.429216 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.429186 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.429275 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.429226 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.429355 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.429286 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.431152 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.431125 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.431410 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.431391 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.437386 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.437365 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzprz\" (UniqueName: \"kubernetes.io/projected/0705b2e8-c8d7-43e3-9393-75e6d83d6e4b-kube-api-access-pzprz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k5d4r\" (UID: \"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.577588 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.577473 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:47.710997 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.710969 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r"] Jul 06 20:24:47.712994 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:24:47.712961 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0705b2e8_c8d7_43e3_9393_75e6d83d6e4b.slice/crio-e83eab7f2e88f48def7eb0ea4459b2145fec8a7c14dd26841a8c8cca142e6351 WatchSource:0}: Error finding container e83eab7f2e88f48def7eb0ea4459b2145fec8a7c14dd26841a8c8cca142e6351: Status 404 returned error can't find the container with id e83eab7f2e88f48def7eb0ea4459b2145fec8a7c14dd26841a8c8cca142e6351 Jul 06 20:24:47.714377 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:47.714360 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:24:48.461630 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:48.461590 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" event={"ID":"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b","Type":"ContainerStarted","Data":"cfa1ecc478c432229b605bb5ce1753302524b6517c2fdd55292c846072fa0927"} Jul 06 20:24:48.462075 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:48.461658 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" event={"ID":"0705b2e8-c8d7-43e3-9393-75e6d83d6e4b","Type":"ContainerStarted","Data":"e83eab7f2e88f48def7eb0ea4459b2145fec8a7c14dd26841a8c8cca142e6351"} Jul 06 20:24:49.649330 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:49.649279 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" podStartSLOduration=2.490041753 podStartE2EDuration="2.649261668s" totalImagesPullingTime="159.219915ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:24:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:48.496223012 +0000 UTC m=+625.342087717" watchObservedRunningTime="2026-07-06 20:24:49.649261668 +0000 UTC m=+626.495126373" Jul 06 20:24:49.656945 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:49.656913 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:24:53.135659 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.135623 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2"] Jul 06 20:24:53.139529 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.139509 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.142008 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.141991 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 06 20:24:53.151369 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.151349 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2"] Jul 06 20:24:53.170681 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.170657 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.170796 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.170698 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.170796 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.170733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cxst\" (UniqueName: \"kubernetes.io/projected/9cb12f48-9fd3-4950-9df9-821a501faff3-kube-api-access-5cxst\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.170796 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.170765 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cb12f48-9fd3-4950-9df9-821a501faff3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.170924 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.170813 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.170924 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.170841 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.271900 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.271862 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cxst\" (UniqueName: \"kubernetes.io/projected/9cb12f48-9fd3-4950-9df9-821a501faff3-kube-api-access-5cxst\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.272059 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.271920 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cb12f48-9fd3-4950-9df9-821a501faff3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.272059 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.271951 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.272059 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.271976 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.272059 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.272045 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.272268 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.272088 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.272426 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.272402 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.272486 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.272443 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.272534 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.272510 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.274351 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.274330 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cb12f48-9fd3-4950-9df9-821a501faff3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.274522 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.274506 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cb12f48-9fd3-4950-9df9-821a501faff3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.281844 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.281819 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cxst\" (UniqueName: \"kubernetes.io/projected/9cb12f48-9fd3-4950-9df9-821a501faff3-kube-api-access-5cxst\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2\" (UID: \"9cb12f48-9fd3-4950-9df9-821a501faff3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.449407 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.449317 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:24:53.581248 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:53.581224 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2"] Jul 06 20:24:53.583845 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:24:53.583812 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cb12f48_9fd3_4950_9df9_821a501faff3.slice/crio-d6e38655e4f44037f0602f59c09e6ffa1acb22a59d01ccf4321ce4a476ecacf3 WatchSource:0}: Error finding container d6e38655e4f44037f0602f59c09e6ffa1acb22a59d01ccf4321ce4a476ecacf3: Status 404 returned error can't find the container with id d6e38655e4f44037f0602f59c09e6ffa1acb22a59d01ccf4321ce4a476ecacf3 Jul 06 20:24:54.490849 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:54.490811 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" event={"ID":"9cb12f48-9fd3-4950-9df9-821a501faff3","Type":"ContainerStarted","Data":"c3625d92154b35282ada59b03004f877eb78e67ea6ac373ab50f92b7e3efee8f"} Jul 06 20:24:54.491236 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:54.490861 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" event={"ID":"9cb12f48-9fd3-4950-9df9-821a501faff3","Type":"ContainerStarted","Data":"d6e38655e4f44037f0602f59c09e6ffa1acb22a59d01ccf4321ce4a476ecacf3"} Jul 06 20:24:54.510767 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:54.510717 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" podStartSLOduration=1.347130534 podStartE2EDuration="1.510700894s" totalImagesPullingTime="163.57036ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:24:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:54.508946617 +0000 UTC m=+631.354811334" watchObservedRunningTime="2026-07-06 20:24:54.510700894 +0000 UTC m=+631.356565598" Jul 06 20:24:56.035541 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:56.035505 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:24:57.578314 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:57.578270 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:57.578840 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:57.578353 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:57.591008 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:57.590984 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:24:58.521421 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:24:58.521382 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k5d4r" Jul 06 20:25:03.449625 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:25:03.449575 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:25:03.449625 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:25:03.449637 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:25:03.462762 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:25:03.462739 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:25:03.544354 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:25:03.544328 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2" Jul 06 20:26:37.548797 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:26:37.548714 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:26:47.250236 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:26:47.250188 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:26:59.368401 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:26:59.368362 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:27:08.158373 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:27:08.158334 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:27:18.475899 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:27:18.475860 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:27:59.064884 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:27:59.064800 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:28:11.150121 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:28:11.150087 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:28:46.148343 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:28:46.148303 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:28:56.138875 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:28:56.138837 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:29:04.172039 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:29:04.171997 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:29:13.360828 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:29:13.360796 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:29:23.454322 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:29:23.454270 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:29:23.591261 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:29:23.591233 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:29:23.596232 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:29:23.596202 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:29:34.047318 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:29:34.047287 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:29:45.050911 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:29:45.050873 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:29:54.263318 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:29:54.263280 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:30:17.453670 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:30:17.453637 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:30:28.880772 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:30:28.880736 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:30:55.564853 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:30:55.564762 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:31:06.046060 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:31:06.046023 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:31:21.656612 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:31:21.656573 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:31:32.539715 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:31:32.539673 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:31:41.642064 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:31:41.642027 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:31:50.861125 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:31:50.861093 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:31:59.237007 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:31:59.236955 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:33:49.841229 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:33:49.841143 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:34:01.136853 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:34:01.136810 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:34:10.244504 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:34:10.244467 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:34:21.669328 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:34:21.669295 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:34:23.620121 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:34:23.620090 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:34:23.623642 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:34:23.623618 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:34:32.462861 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:34:32.462826 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:34:43.283603 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:34:43.283566 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:34:52.950839 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:34:52.950799 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:35:03.270081 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:35:03.270043 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:35:14.944091 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:35:14.944051 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:35:24.161090 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:35:24.161008 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:35:34.248029 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:35:34.247990 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:35:58.785968 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:35:58.785926 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:36:05.765865 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:36:05.765830 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:36:15.447441 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:36:15.447403 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:36:26.736855 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:36:26.736817 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:36:34.138862 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:36:34.138826 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:36:46.457543 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:36:46.457464 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:37:02.342204 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:37:02.342163 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:37:10.742751 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:37:10.742716 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:37:16.149654 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:37:16.149614 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:37:25.251840 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:37:25.251801 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:37:34.059965 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:37:34.059927 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:37:39.352976 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:37:39.352940 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:37:48.648231 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:37:48.648195 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:37:57.553127 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:37:57.553091 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:38:05.847827 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:38:05.847790 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:38:17.445170 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:38:17.445084 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:38:26.858530 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:38:26.858494 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:38:33.543282 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:38:33.543250 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:38:48.247992 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:38:48.247957 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:38:59.551338 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:38:59.551301 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:39:11.844347 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:39:11.844314 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:39:23.652694 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:39:23.652662 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:39:23.653356 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:39:23.653338 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:39:24.541865 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:39:24.541836 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:39:32.834489 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:39:32.834447 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:39:44.753371 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:39:44.753338 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:39:51.447485 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:39:51.447443 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:39:57.569303 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:39:57.569262 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:40:19.342781 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:40:19.342739 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:40:29.239208 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:40:29.239145 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:41:51.151438 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:41:51.151347 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:42:00.345599 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:42:00.345526 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:42:21.138412 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:42:21.138378 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:42:33.633451 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:42:33.633413 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:43:04.762691 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:43:04.762597 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:43:31.874473 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:43:31.874432 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:43:56.640790 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:43:56.640746 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:44:06.784676 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:44:06.784633 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:44:16.063371 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:44:16.063280 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:44:23.682222 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:44:23.682191 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:44:23.683603 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:44:23.683581 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:44:52.943148 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:44:52.943114 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:45:00.180721 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:00.180675 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722845-88cxv"] Jul 06 20:45:00.184252 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:00.184226 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722845-88cxv" Jul 06 20:45:00.186932 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:00.186910 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5t798\"" Jul 06 20:45:00.194804 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:00.194780 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722845-88cxv"] Jul 06 20:45:00.277188 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:00.277129 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4rdr\" (UniqueName: \"kubernetes.io/projected/641b865d-fa84-469d-a769-7b2b32cf207b-kube-api-access-f4rdr\") pod \"maas-api-key-cleanup-29722845-88cxv\" (UID: \"641b865d-fa84-469d-a769-7b2b32cf207b\") " pod="opendatahub/maas-api-key-cleanup-29722845-88cxv" Jul 06 20:45:00.378353 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:00.378309 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4rdr\" (UniqueName: \"kubernetes.io/projected/641b865d-fa84-469d-a769-7b2b32cf207b-kube-api-access-f4rdr\") pod \"maas-api-key-cleanup-29722845-88cxv\" (UID: \"641b865d-fa84-469d-a769-7b2b32cf207b\") " pod="opendatahub/maas-api-key-cleanup-29722845-88cxv" Jul 06 20:45:00.386709 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:00.386679 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4rdr\" (UniqueName: \"kubernetes.io/projected/641b865d-fa84-469d-a769-7b2b32cf207b-kube-api-access-f4rdr\") pod \"maas-api-key-cleanup-29722845-88cxv\" (UID: \"641b865d-fa84-469d-a769-7b2b32cf207b\") " pod="opendatahub/maas-api-key-cleanup-29722845-88cxv" Jul 06 20:45:00.495503 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:00.495468 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722845-88cxv" Jul 06 20:45:00.632114 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:00.632073 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722845-88cxv"] Jul 06 20:45:00.634434 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:45:00.634396 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641b865d_fa84_469d_a769_7b2b32cf207b.slice/crio-dea5b40e79ac862e727836d52373ce52956f99b2b86fdfc0c0fa8e1ffccc128b WatchSource:0}: Error finding container dea5b40e79ac862e727836d52373ce52956f99b2b86fdfc0c0fa8e1ffccc128b: Status 404 returned error can't find the container with id dea5b40e79ac862e727836d52373ce52956f99b2b86fdfc0c0fa8e1ffccc128b Jul 06 20:45:00.636435 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:00.636419 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:45:01.081247 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:01.081212 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722845-88cxv" event={"ID":"641b865d-fa84-469d-a769-7b2b32cf207b","Type":"ContainerStarted","Data":"dea5b40e79ac862e727836d52373ce52956f99b2b86fdfc0c0fa8e1ffccc128b"} Jul 06 20:45:03.092697 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:03.092663 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722845-88cxv" event={"ID":"641b865d-fa84-469d-a769-7b2b32cf207b","Type":"ContainerStarted","Data":"6e4b4d6cfd39399ccdbb8aededb400a60693e6722c050e41a22b9a44b7bce4f0"} Jul 06 20:45:04.099206 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:04.099170 2566 generic.go:356] "Generic (PLEG): container finished" podID="641b865d-fa84-469d-a769-7b2b32cf207b" containerID="6e4b4d6cfd39399ccdbb8aededb400a60693e6722c050e41a22b9a44b7bce4f0" exitCode=0 Jul 06 20:45:04.099678 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:04.099214 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722845-88cxv" event={"ID":"641b865d-fa84-469d-a769-7b2b32cf207b","Type":"ContainerDied","Data":"6e4b4d6cfd39399ccdbb8aededb400a60693e6722c050e41a22b9a44b7bce4f0"} Jul 06 20:45:05.233563 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:05.233523 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722845-88cxv" Jul 06 20:45:05.322773 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:05.322734 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4rdr\" (UniqueName: \"kubernetes.io/projected/641b865d-fa84-469d-a769-7b2b32cf207b-kube-api-access-f4rdr\") pod \"641b865d-fa84-469d-a769-7b2b32cf207b\" (UID: \"641b865d-fa84-469d-a769-7b2b32cf207b\") " Jul 06 20:45:05.324944 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:05.324917 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/641b865d-fa84-469d-a769-7b2b32cf207b-kube-api-access-f4rdr" (OuterVolumeSpecName: "kube-api-access-f4rdr") pod "641b865d-fa84-469d-a769-7b2b32cf207b" (UID: "641b865d-fa84-469d-a769-7b2b32cf207b"). InnerVolumeSpecName "kube-api-access-f4rdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:45:05.424176 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:05.424078 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4rdr\" (UniqueName: \"kubernetes.io/projected/641b865d-fa84-469d-a769-7b2b32cf207b-kube-api-access-f4rdr\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:45:06.109689 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:06.109653 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722845-88cxv" event={"ID":"641b865d-fa84-469d-a769-7b2b32cf207b","Type":"ContainerDied","Data":"dea5b40e79ac862e727836d52373ce52956f99b2b86fdfc0c0fa8e1ffccc128b"} Jul 06 20:45:06.109689 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:06.109677 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722845-88cxv" Jul 06 20:45:06.109896 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:06.109701 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dea5b40e79ac862e727836d52373ce52956f99b2b86fdfc0c0fa8e1ffccc128b" Jul 06 20:45:24.073777 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:24.073742 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:45:37.762276 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:37.762236 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:45:45.252119 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:45:45.252085 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:46:33.367837 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:46:33.365275 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:46:46.349383 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:46:46.349340 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:47:02.049843 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:02.049806 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:47:16.954530 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:16.954455 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn"] Jul 06 20:47:16.955026 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:16.955009 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641b865d-fa84-469d-a769-7b2b32cf207b" containerName="cleanup" Jul 06 20:47:16.955072 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:16.955029 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="641b865d-fa84-469d-a769-7b2b32cf207b" containerName="cleanup" Jul 06 20:47:16.955131 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:16.955121 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="641b865d-fa84-469d-a769-7b2b32cf207b" containerName="cleanup" Jul 06 20:47:16.958599 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:16.958579 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:16.962204 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:16.962185 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-07298a\"/\"kube-root-ca.crt\"" Jul 06 20:47:16.963310 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:16.963290 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-07298a\"/\"default-dockercfg-8w74r\"" Jul 06 20:47:16.963388 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:16.963298 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-07298a\"/\"test-model-07298a-kserve-self-signed-certs\"" Jul 06 20:47:16.963388 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:16.963338 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-07298a\"/\"openshift-service-ca.crt\"" Jul 06 20:47:16.969917 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:16.969892 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn"] Jul 06 20:47:17.055533 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.055503 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36d8ade3-a682-4b52-bf1e-882ac02c0927-tls-certs\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.055658 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.055562 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-home\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.055658 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.055630 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-dshm\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.055750 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.055658 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-tmp-dir\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.055750 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.055681 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-model-cache\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.055750 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.055704 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glqgr\" (UniqueName: \"kubernetes.io/projected/36d8ade3-a682-4b52-bf1e-882ac02c0927-kube-api-access-glqgr\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.156230 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.156195 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36d8ade3-a682-4b52-bf1e-882ac02c0927-tls-certs\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.156334 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.156246 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-home\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.156334 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.156278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-dshm\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.156408 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.156367 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-tmp-dir\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.156441 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.156408 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-model-cache\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.156481 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.156440 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glqgr\" (UniqueName: \"kubernetes.io/projected/36d8ade3-a682-4b52-bf1e-882ac02c0927-kube-api-access-glqgr\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.156750 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.156730 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-home\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.156837 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.156761 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-tmp-dir\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.156837 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.156802 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-model-cache\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.158464 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.158444 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-dshm\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.158635 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.158620 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36d8ade3-a682-4b52-bf1e-882ac02c0927-tls-certs\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.165530 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.165507 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glqgr\" (UniqueName: \"kubernetes.io/projected/36d8ade3-a682-4b52-bf1e-882ac02c0927-kube-api-access-glqgr\") pod \"test-model-07298a-kserve-7b7dcf57f9-nxkmn\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.268686 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.268661 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:17.401586 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.401560 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn"] Jul 06 20:47:17.403502 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:47:17.403471 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36d8ade3_a682_4b52_bf1e_882ac02c0927.slice/crio-cb0d4a6dbe155e9199fa545339fda0227b295e34470713c285cd6befedbe0a71 WatchSource:0}: Error finding container cb0d4a6dbe155e9199fa545339fda0227b295e34470713c285cd6befedbe0a71: Status 404 returned error can't find the container with id cb0d4a6dbe155e9199fa545339fda0227b295e34470713c285cd6befedbe0a71 Jul 06 20:47:17.743009 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.742970 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" event={"ID":"36d8ade3-a682-4b52-bf1e-882ac02c0927","Type":"ContainerStarted","Data":"c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c"} Jul 06 20:47:17.743009 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.743012 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" event={"ID":"36d8ade3-a682-4b52-bf1e-882ac02c0927","Type":"ContainerStarted","Data":"cb0d4a6dbe155e9199fa545339fda0227b295e34470713c285cd6befedbe0a71"} Jul 06 20:47:17.768487 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:17.768449 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" podStartSLOduration=1.7684358919999998 podStartE2EDuration="1.768435892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:47:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:47:17.767669066 +0000 UTC m=+1974.613533770" watchObservedRunningTime="2026-07-06 20:47:17.768435892 +0000 UTC m=+1974.614300596" Jul 06 20:47:20.957236 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:20.957204 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:47:27.269817 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:27.269772 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:27.270226 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:27.270067 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:27.282583 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:27.282540 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:27.806242 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:27.806215 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:33.842303 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:33.842269 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:47:37.382945 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:37.382907 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn"] Jul 06 20:47:48.549974 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:48.549942 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:47:52.402533 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.402492 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" podUID="36d8ade3-a682-4b52-bf1e-882ac02c0927" containerName="main" containerID="cri-o://c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c" gracePeriod=45 Jul 06 20:47:52.651204 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.651177 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:52.758901 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.758870 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-home\") pod \"36d8ade3-a682-4b52-bf1e-882ac02c0927\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " Jul 06 20:47:52.759080 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.758909 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-model-cache\") pod \"36d8ade3-a682-4b52-bf1e-882ac02c0927\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " Jul 06 20:47:52.759080 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.758942 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36d8ade3-a682-4b52-bf1e-882ac02c0927-tls-certs\") pod \"36d8ade3-a682-4b52-bf1e-882ac02c0927\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " Jul 06 20:47:52.759080 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.759033 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-dshm\") pod \"36d8ade3-a682-4b52-bf1e-882ac02c0927\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " Jul 06 20:47:52.759080 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.759057 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-tmp-dir\") pod \"36d8ade3-a682-4b52-bf1e-882ac02c0927\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " Jul 06 20:47:52.759331 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.759086 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glqgr\" (UniqueName: \"kubernetes.io/projected/36d8ade3-a682-4b52-bf1e-882ac02c0927-kube-api-access-glqgr\") pod \"36d8ade3-a682-4b52-bf1e-882ac02c0927\" (UID: \"36d8ade3-a682-4b52-bf1e-882ac02c0927\") " Jul 06 20:47:52.759331 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.759200 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-model-cache" (OuterVolumeSpecName: "model-cache") pod "36d8ade3-a682-4b52-bf1e-882ac02c0927" (UID: "36d8ade3-a682-4b52-bf1e-882ac02c0927"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:47:52.759331 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.759223 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-home" (OuterVolumeSpecName: "home") pod "36d8ade3-a682-4b52-bf1e-882ac02c0927" (UID: "36d8ade3-a682-4b52-bf1e-882ac02c0927"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:47:52.759433 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.759330 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36d8ade3-a682-4b52-bf1e-882ac02c0927" (UID: "36d8ade3-a682-4b52-bf1e-882ac02c0927"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:47:52.759470 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.759429 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-tmp-dir\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:47:52.759470 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.759444 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-home\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:47:52.759470 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.759453 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-model-cache\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:47:52.761266 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.761241 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-dshm" (OuterVolumeSpecName: "dshm") pod "36d8ade3-a682-4b52-bf1e-882ac02c0927" (UID: "36d8ade3-a682-4b52-bf1e-882ac02c0927"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:47:52.761378 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.761344 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36d8ade3-a682-4b52-bf1e-882ac02c0927-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36d8ade3-a682-4b52-bf1e-882ac02c0927" (UID: "36d8ade3-a682-4b52-bf1e-882ac02c0927"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:47:52.761378 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.761352 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36d8ade3-a682-4b52-bf1e-882ac02c0927-kube-api-access-glqgr" (OuterVolumeSpecName: "kube-api-access-glqgr") pod "36d8ade3-a682-4b52-bf1e-882ac02c0927" (UID: "36d8ade3-a682-4b52-bf1e-882ac02c0927"). InnerVolumeSpecName "kube-api-access-glqgr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:47:52.860058 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.860026 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36d8ade3-a682-4b52-bf1e-882ac02c0927-tls-certs\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:47:52.860058 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.860051 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36d8ade3-a682-4b52-bf1e-882ac02c0927-dshm\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:47:52.860058 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.860060 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glqgr\" (UniqueName: \"kubernetes.io/projected/36d8ade3-a682-4b52-bf1e-882ac02c0927-kube-api-access-glqgr\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:47:52.911490 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.911439 2566 generic.go:356] "Generic (PLEG): container finished" podID="36d8ade3-a682-4b52-bf1e-882ac02c0927" containerID="c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c" exitCode=0 Jul 06 20:47:52.911490 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.911484 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" Jul 06 20:47:52.911705 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.911529 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" event={"ID":"36d8ade3-a682-4b52-bf1e-882ac02c0927","Type":"ContainerDied","Data":"c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c"} Jul 06 20:47:52.911705 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.911597 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn" event={"ID":"36d8ade3-a682-4b52-bf1e-882ac02c0927","Type":"ContainerDied","Data":"cb0d4a6dbe155e9199fa545339fda0227b295e34470713c285cd6befedbe0a71"} Jul 06 20:47:52.911705 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.911616 2566 scope.go:117] "RemoveContainer" containerID="c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c" Jul 06 20:47:52.920791 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.920776 2566 scope.go:117] "RemoveContainer" containerID="c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c" Jul 06 20:47:52.921023 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:47:52.921004 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c\": container with ID starting with c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c not found: ID does not exist" containerID="c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c" Jul 06 20:47:52.921072 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.921036 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c"} err="failed to get container status \"c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c\": rpc error: code = NotFound desc = could not find container \"c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c\": container with ID starting with c8904249d1ec099cfde32c0d2a860b498af13ef3fac2b70f8fc6351216dca90c not found: ID does not exist" Jul 06 20:47:52.933443 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.933424 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn"] Jul 06 20:47:52.937614 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:52.937592 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-07298a/test-model-07298a-kserve-7b7dcf57f9-nxkmn"] Jul 06 20:47:53.670984 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:47:53.670953 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36d8ade3-a682-4b52-bf1e-882ac02c0927" path="/var/lib/kubelet/pods/36d8ade3-a682-4b52-bf1e-882ac02c0927/volumes" Jul 06 20:48:24.644207 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:48:24.644169 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:48:37.487490 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:48:37.487451 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:48:53.360761 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:48:53.360683 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:49:04.339397 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:49:04.339364 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:49:15.646568 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:49:15.646518 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:49:23.709418 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:49:23.709386 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:49:23.711644 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:49:23.711619 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:49:25.049506 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:49:25.049473 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:49:47.240214 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:49:47.240179 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:50:03.861428 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.861388 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d"] Jul 06 20:50:03.864038 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.861915 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36d8ade3-a682-4b52-bf1e-882ac02c0927" containerName="main" Jul 06 20:50:03.864038 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.861931 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="36d8ade3-a682-4b52-bf1e-882ac02c0927" containerName="main" Jul 06 20:50:03.864038 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.861998 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="36d8ade3-a682-4b52-bf1e-882ac02c0927" containerName="main" Jul 06 20:50:03.864978 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.864960 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:03.867522 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.867504 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d729fa\"/\"kube-root-ca.crt\"" Jul 06 20:50:03.868727 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.868699 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d729fa\"/\"openshift-service-ca.crt\"" Jul 06 20:50:03.868820 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.868761 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d729fa\"/\"auth-test-model-d729fa-kserve-self-signed-certs\"" Jul 06 20:50:03.869222 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.869209 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d729fa\"/\"default-dockercfg-5zgpd\"" Jul 06 20:50:03.876601 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.876580 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d"] Jul 06 20:50:03.960606 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.960579 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tmp-dir\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:03.960701 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.960618 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-dshm\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:03.960701 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.960643 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j9z4\" (UniqueName: \"kubernetes.io/projected/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-kube-api-access-7j9z4\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:03.960774 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.960719 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-model-cache\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:03.960807 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.960786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-home\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:03.960839 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:03.960826 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tls-certs\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.061972 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.061936 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-home\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.062072 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.061984 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tls-certs\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.062126 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.062106 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tmp-dir\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.062164 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.062156 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-dshm\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.062199 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.062186 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j9z4\" (UniqueName: \"kubernetes.io/projected/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-kube-api-access-7j9z4\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.062235 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.062215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-model-cache\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.062360 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.062329 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-home\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.062500 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.062394 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tmp-dir\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.062657 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.062635 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-model-cache\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.064295 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.064275 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-dshm\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.064435 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.064419 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tls-certs\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.073273 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.073253 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j9z4\" (UniqueName: \"kubernetes.io/projected/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-kube-api-access-7j9z4\") pod \"auth-test-model-d729fa-kserve-68c77b46dc-8gj5d\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.173697 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.173636 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:04.307382 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.307350 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d"] Jul 06 20:50:04.310094 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:50:04.310070 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod633c9fc8_c4bc_4c32_b9c3_84afd80664b3.slice/crio-6bf5b07fcf94251884c5b9e290f5a2193887fd0aa31fc8f3e4a80f7ffd25381c WatchSource:0}: Error finding container 6bf5b07fcf94251884c5b9e290f5a2193887fd0aa31fc8f3e4a80f7ffd25381c: Status 404 returned error can't find the container with id 6bf5b07fcf94251884c5b9e290f5a2193887fd0aa31fc8f3e4a80f7ffd25381c Jul 06 20:50:04.510394 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.510364 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" event={"ID":"633c9fc8-c4bc-4c32-b9c3-84afd80664b3","Type":"ContainerStarted","Data":"291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8"} Jul 06 20:50:04.510579 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.510403 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" event={"ID":"633c9fc8-c4bc-4c32-b9c3-84afd80664b3","Type":"ContainerStarted","Data":"6bf5b07fcf94251884c5b9e290f5a2193887fd0aa31fc8f3e4a80f7ffd25381c"} Jul 06 20:50:04.530872 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:04.530828 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" podStartSLOduration=1.530814216 podStartE2EDuration="1.530814216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:50:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:50:04.529462007 +0000 UTC m=+2141.375326712" watchObservedRunningTime="2026-07-06 20:50:04.530814216 +0000 UTC m=+2141.376678898" Jul 06 20:50:07.159105 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:07.159067 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:50:14.174163 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:14.174129 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:14.174626 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:14.174178 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:14.186396 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:14.186370 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:14.570744 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:14.570719 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:50:20.662006 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:20.661912 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:50:35.744508 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:35.744477 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:50:51.544867 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:50:51.544833 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:51:05.141408 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:05.141373 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:51:20.340612 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:20.340571 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:51:37.446320 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:37.446285 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:51:50.431485 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:50.431408 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d"] Jul 06 20:51:54.437883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.437851 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx"] Jul 06 20:51:54.441163 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.441147 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.443883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.443858 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7306a9\"/\"kube-root-ca.crt\"" Jul 06 20:51:54.444024 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.443899 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7306a9\"/\"openshift-service-ca.crt\"" Jul 06 20:51:54.444847 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.444829 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7306a9\"/\"sub-test-model-7306a9-kserve-self-signed-certs\"" Jul 06 20:51:54.444928 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.444890 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7306a9\"/\"default-dockercfg-6vkvc\"" Jul 06 20:51:54.455620 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.455599 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx"] Jul 06 20:51:54.548261 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.548233 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:51:54.564281 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.564261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-home\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.564388 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.564290 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tmp-dir\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.564388 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.564318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-dshm\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.564388 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.564341 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tls-certs\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.564388 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.564367 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l5kc\" (UniqueName: \"kubernetes.io/projected/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-kube-api-access-2l5kc\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.564539 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.564402 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-model-cache\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.664848 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.664822 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-dshm\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.664961 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.664855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tls-certs\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.664961 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.664873 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l5kc\" (UniqueName: \"kubernetes.io/projected/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-kube-api-access-2l5kc\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.664961 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.664889 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-model-cache\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.665103 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.664999 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-home\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.665103 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.665024 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tmp-dir\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.665322 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.665299 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-model-cache\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.665473 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.665368 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-home\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.665596 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.665519 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tmp-dir\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.667211 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.667192 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-dshm\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.667309 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.667298 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tls-certs\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.674916 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.674893 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l5kc\" (UniqueName: \"kubernetes.io/projected/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-kube-api-access-2l5kc\") pod \"sub-test-model-7306a9-kserve-7674d8fc55-h6nvx\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.750243 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.750219 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:51:54.890171 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:54.890146 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx"] Jul 06 20:51:54.892081 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:51:54.892048 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f8a7e58_4c34_4932_b9dc_baa0fae2a2c9.slice/crio-96f06931c13f4e70b61c9cde7ed6844d91490b2a992e241b57039c9d53681f98 WatchSource:0}: Error finding container 96f06931c13f4e70b61c9cde7ed6844d91490b2a992e241b57039c9d53681f98: Status 404 returned error can't find the container with id 96f06931c13f4e70b61c9cde7ed6844d91490b2a992e241b57039c9d53681f98 Jul 06 20:51:55.019460 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:55.019377 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" event={"ID":"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9","Type":"ContainerStarted","Data":"3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de"} Jul 06 20:51:55.019460 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:55.019424 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" event={"ID":"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9","Type":"ContainerStarted","Data":"96f06931c13f4e70b61c9cde7ed6844d91490b2a992e241b57039c9d53681f98"} Jul 06 20:51:55.048005 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:51:55.047960 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" podStartSLOduration=1.0479470850000001 podStartE2EDuration="1.047947085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:51:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:51:55.04658485 +0000 UTC m=+2251.892449548" watchObservedRunningTime="2026-07-06 20:51:55.047947085 +0000 UTC m=+2251.893811815" Jul 06 20:52:04.751225 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:04.751191 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:52:04.751621 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:04.751234 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:52:04.763249 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:04.763227 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:52:05.077084 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.077004 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:52:05.451037 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.450947 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" podUID="633c9fc8-c4bc-4c32-b9c3-84afd80664b3" containerName="main" containerID="cri-o://291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8" gracePeriod=45 Jul 06 20:52:05.701956 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.701902 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:52:05.863563 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.863529 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tmp-dir\") pod \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " Jul 06 20:52:05.863999 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.863574 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-model-cache\") pod \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " Jul 06 20:52:05.863999 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.863597 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j9z4\" (UniqueName: \"kubernetes.io/projected/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-kube-api-access-7j9z4\") pod \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " Jul 06 20:52:05.863999 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.863628 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-home\") pod \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " Jul 06 20:52:05.863999 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.863685 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-dshm\") pod \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " Jul 06 20:52:05.863999 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.863746 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tls-certs\") pod \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\" (UID: \"633c9fc8-c4bc-4c32-b9c3-84afd80664b3\") " Jul 06 20:52:05.863999 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.863796 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-model-cache" (OuterVolumeSpecName: "model-cache") pod "633c9fc8-c4bc-4c32-b9c3-84afd80664b3" (UID: "633c9fc8-c4bc-4c32-b9c3-84afd80664b3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:52:05.863999 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.863814 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "633c9fc8-c4bc-4c32-b9c3-84afd80664b3" (UID: "633c9fc8-c4bc-4c32-b9c3-84afd80664b3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:52:05.863999 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.863901 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-home" (OuterVolumeSpecName: "home") pod "633c9fc8-c4bc-4c32-b9c3-84afd80664b3" (UID: "633c9fc8-c4bc-4c32-b9c3-84afd80664b3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:52:05.864329 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.864021 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-home\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:52:05.864329 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.864040 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tmp-dir\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:52:05.864329 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.864054 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-model-cache\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:52:05.865966 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.865939 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "633c9fc8-c4bc-4c32-b9c3-84afd80664b3" (UID: "633c9fc8-c4bc-4c32-b9c3-84afd80664b3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:52:05.866069 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.865997 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-kube-api-access-7j9z4" (OuterVolumeSpecName: "kube-api-access-7j9z4") pod "633c9fc8-c4bc-4c32-b9c3-84afd80664b3" (UID: "633c9fc8-c4bc-4c32-b9c3-84afd80664b3"). InnerVolumeSpecName "kube-api-access-7j9z4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:52:05.866069 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.866009 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-dshm" (OuterVolumeSpecName: "dshm") pod "633c9fc8-c4bc-4c32-b9c3-84afd80664b3" (UID: "633c9fc8-c4bc-4c32-b9c3-84afd80664b3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:52:05.964972 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.964907 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-tls-certs\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:52:05.964972 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.964932 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j9z4\" (UniqueName: \"kubernetes.io/projected/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-kube-api-access-7j9z4\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:52:05.964972 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:05.964942 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633c9fc8-c4bc-4c32-b9c3-84afd80664b3-dshm\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:52:06.068876 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:06.068845 2566 generic.go:356] "Generic (PLEG): container finished" podID="633c9fc8-c4bc-4c32-b9c3-84afd80664b3" containerID="291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8" exitCode=0 Jul 06 20:52:06.069031 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:06.068887 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" Jul 06 20:52:06.069031 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:06.068920 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" event={"ID":"633c9fc8-c4bc-4c32-b9c3-84afd80664b3","Type":"ContainerDied","Data":"291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8"} Jul 06 20:52:06.069031 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:06.068961 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d" event={"ID":"633c9fc8-c4bc-4c32-b9c3-84afd80664b3","Type":"ContainerDied","Data":"6bf5b07fcf94251884c5b9e290f5a2193887fd0aa31fc8f3e4a80f7ffd25381c"} Jul 06 20:52:06.069031 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:06.068980 2566 scope.go:117] "RemoveContainer" containerID="291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8" Jul 06 20:52:06.080390 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:06.080374 2566 scope.go:117] "RemoveContainer" containerID="291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8" Jul 06 20:52:06.080666 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:52:06.080647 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8\": container with ID starting with 291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8 not found: ID does not exist" containerID="291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8" Jul 06 20:52:06.080733 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:06.080673 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8"} err="failed to get container status \"291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8\": rpc error: code = NotFound desc = could not find container \"291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8\": container with ID starting with 291637a86890f70e7d71bc5a7a5cf78a27ad207aabda0449388d55d5a8a675f8 not found: ID does not exist" Jul 06 20:52:06.092439 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:06.092414 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d"] Jul 06 20:52:06.098240 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:06.098219 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/auth-test-model-d729fa-kserve-68c77b46dc-8gj5d"] Jul 06 20:52:07.668730 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:07.668697 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="633c9fc8-c4bc-4c32-b9c3-84afd80664b3" path="/var/lib/kubelet/pods/633c9fc8-c4bc-4c32-b9c3-84afd80664b3/volumes" Jul 06 20:52:09.260726 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:09.260688 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:52:19.774200 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:19.774169 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:52:33.913881 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:33.913840 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:52:50.021057 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:50.021019 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx"] Jul 06 20:52:54.358112 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:52:54.358080 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:53:05.040927 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.040859 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" podUID="9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" containerName="main" containerID="cri-o://3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de" gracePeriod=45 Jul 06 20:53:05.064612 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.064582 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" podUID="9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": dial tcp 10.132.0.41:8000: connect: connection refused" Jul 06 20:53:05.290369 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.290344 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:53:05.353588 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.353496 2566 generic.go:356] "Generic (PLEG): container finished" podID="9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" containerID="3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de" exitCode=0 Jul 06 20:53:05.353588 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.353535 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" Jul 06 20:53:05.353788 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.353577 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" event={"ID":"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9","Type":"ContainerDied","Data":"3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de"} Jul 06 20:53:05.353788 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.353627 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx" event={"ID":"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9","Type":"ContainerDied","Data":"96f06931c13f4e70b61c9cde7ed6844d91490b2a992e241b57039c9d53681f98"} Jul 06 20:53:05.353788 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.353651 2566 scope.go:117] "RemoveContainer" containerID="3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de" Jul 06 20:53:05.362088 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.362071 2566 scope.go:117] "RemoveContainer" containerID="3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de" Jul 06 20:53:05.362350 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:53:05.362333 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de\": container with ID starting with 3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de not found: ID does not exist" containerID="3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de" Jul 06 20:53:05.362411 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.362361 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de"} err="failed to get container status \"3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de\": rpc error: code = NotFound desc = could not find container \"3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de\": container with ID starting with 3392b8ddf6bf14e25cc72845f7931cb454824f797cf1c7fc7629ab98e843e9de not found: ID does not exist" Jul 06 20:53:05.370125 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.370110 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-home\") pod \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " Jul 06 20:53:05.370176 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.370147 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tls-certs\") pod \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " Jul 06 20:53:05.370213 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.370201 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tmp-dir\") pod \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " Jul 06 20:53:05.370249 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.370219 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l5kc\" (UniqueName: \"kubernetes.io/projected/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-kube-api-access-2l5kc\") pod \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " Jul 06 20:53:05.370291 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.370279 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-dshm\") pod \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " Jul 06 20:53:05.370331 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.370304 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-model-cache\") pod \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\" (UID: \"9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9\") " Jul 06 20:53:05.370386 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.370336 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-home" (OuterVolumeSpecName: "home") pod "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" (UID: "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:53:05.370477 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.370457 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" (UID: "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:53:05.370573 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.370533 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-home\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:53:05.370632 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.370614 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tmp-dir\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:53:05.370683 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.370650 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" (UID: "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:53:05.372461 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.372433 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-dshm" (OuterVolumeSpecName: "dshm") pod "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" (UID: "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:53:05.372461 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.372438 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" (UID: "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:53:05.372608 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.372483 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-kube-api-access-2l5kc" (OuterVolumeSpecName: "kube-api-access-2l5kc") pod "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" (UID: "9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9"). InnerVolumeSpecName "kube-api-access-2l5kc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:53:05.471877 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.471847 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-model-cache\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:53:05.471877 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.471871 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-tls-certs\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:53:05.471877 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.471883 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l5kc\" (UniqueName: \"kubernetes.io/projected/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-kube-api-access-2l5kc\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:53:05.472086 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.471892 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9-dshm\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:53:05.678668 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.678642 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx"] Jul 06 20:53:05.686631 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:05.686609 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7306a9/sub-test-model-7306a9-kserve-7674d8fc55-h6nvx"] Jul 06 20:53:07.668511 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:07.668477 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" path="/var/lib/kubelet/pods/9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9/volumes" Jul 06 20:53:15.143178 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:15.143141 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:53:40.240273 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.240227 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986"] Jul 06 20:53:40.240752 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.240685 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" containerName="main" Jul 06 20:53:40.240752 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.240703 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" containerName="main" Jul 06 20:53:40.240752 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.240745 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="633c9fc8-c4bc-4c32-b9c3-84afd80664b3" containerName="main" Jul 06 20:53:40.240752 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.240753 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="633c9fc8-c4bc-4c32-b9c3-84afd80664b3" containerName="main" Jul 06 20:53:40.240951 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.240850 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8a7e58-4c34-4932-b9dc-baa0fae2a2c9" containerName="main" Jul 06 20:53:40.240951 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.240866 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="633c9fc8-c4bc-4c32-b9c3-84afd80664b3" containerName="main" Jul 06 20:53:40.244101 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.244084 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.247847 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.247824 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d729fa\"/\"openshift-service-ca.crt\"" Jul 06 20:53:40.247847 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.247834 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d729fa\"/\"default-dockercfg-5zgpd\"" Jul 06 20:53:40.248034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.247864 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d729fa\"/\"kube-root-ca.crt\"" Jul 06 20:53:40.248034 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.247878 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d729fa\"/\"rate-test-model-d729fa-kserve-self-signed-certs\"" Jul 06 20:53:40.253513 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.253491 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986"] Jul 06 20:53:40.370205 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.370161 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqnhs\" (UniqueName: \"kubernetes.io/projected/2a9addf2-5426-4def-bae1-d5019f86c95e-kube-api-access-hqnhs\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.370205 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.370211 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-dshm\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.370459 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.370238 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-tmp-dir\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.370459 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.370297 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-model-cache\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.370459 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.370359 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a9addf2-5426-4def-bae1-d5019f86c95e-tls-certs\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.370459 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.370428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-home\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.471412 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.471362 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-home\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.471639 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.471432 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqnhs\" (UniqueName: \"kubernetes.io/projected/2a9addf2-5426-4def-bae1-d5019f86c95e-kube-api-access-hqnhs\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.471639 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.471449 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-dshm\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.471639 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.471464 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-tmp-dir\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.471639 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.471487 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-model-cache\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.471639 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.471515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a9addf2-5426-4def-bae1-d5019f86c95e-tls-certs\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.471883 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.471865 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-home\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.471944 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.471889 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-model-cache\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.472068 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.472045 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-tmp-dir\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.473788 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.473760 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-dshm\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.474028 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.474008 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a9addf2-5426-4def-bae1-d5019f86c95e-tls-certs\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.480976 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.480944 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqnhs\" (UniqueName: \"kubernetes.io/projected/2a9addf2-5426-4def-bae1-d5019f86c95e-kube-api-access-hqnhs\") pod \"rate-test-model-d729fa-kserve-555c6586b7-45986\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.554889 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.554784 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:40.705752 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:40.705723 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986"] Jul 06 20:53:40.707171 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:53:40.707141 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a9addf2_5426_4def_bae1_d5019f86c95e.slice/crio-32aa9734d52925115c39b25281f91262577e2cc9e58b5caaceab63daab52e0a4 WatchSource:0}: Error finding container 32aa9734d52925115c39b25281f91262577e2cc9e58b5caaceab63daab52e0a4: Status 404 returned error can't find the container with id 32aa9734d52925115c39b25281f91262577e2cc9e58b5caaceab63daab52e0a4 Jul 06 20:53:41.528803 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:41.528763 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" event={"ID":"2a9addf2-5426-4def-bae1-d5019f86c95e","Type":"ContainerStarted","Data":"8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf"} Jul 06 20:53:41.528803 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:41.528810 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" event={"ID":"2a9addf2-5426-4def-bae1-d5019f86c95e","Type":"ContainerStarted","Data":"32aa9734d52925115c39b25281f91262577e2cc9e58b5caaceab63daab52e0a4"} Jul 06 20:53:41.544915 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:41.544860 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" podStartSLOduration=1.544841911 podStartE2EDuration="1.544841911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:53:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:53:41.544697783 +0000 UTC m=+2358.390562489" watchObservedRunningTime="2026-07-06 20:53:41.544841911 +0000 UTC m=+2358.390706616" Jul 06 20:53:44.464295 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:44.464251 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:53:50.555600 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:50.555526 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:50.555600 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:50.555609 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:50.568455 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:50.568425 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:53:50.586699 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:53:50.586666 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:54:10.937409 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:10.937367 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:54:23.255656 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:23.255611 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:54:23.737610 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:23.737506 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:54:23.740641 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:23.740618 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:54:31.525958 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:31.525913 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986"] Jul 06 20:54:36.336450 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:36.336410 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:54:46.546751 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.546641 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" podUID="2a9addf2-5426-4def-bae1-d5019f86c95e" containerName="main" containerID="cri-o://8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf" gracePeriod=45 Jul 06 20:54:46.794617 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.794589 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:54:46.847212 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.847140 2566 generic.go:356] "Generic (PLEG): container finished" podID="2a9addf2-5426-4def-bae1-d5019f86c95e" containerID="8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf" exitCode=0 Jul 06 20:54:46.847212 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.847183 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" Jul 06 20:54:46.847368 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.847229 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" event={"ID":"2a9addf2-5426-4def-bae1-d5019f86c95e","Type":"ContainerDied","Data":"8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf"} Jul 06 20:54:46.847368 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.847275 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986" event={"ID":"2a9addf2-5426-4def-bae1-d5019f86c95e","Type":"ContainerDied","Data":"32aa9734d52925115c39b25281f91262577e2cc9e58b5caaceab63daab52e0a4"} Jul 06 20:54:46.847368 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.847296 2566 scope.go:117] "RemoveContainer" containerID="8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf" Jul 06 20:54:46.855743 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.855724 2566 scope.go:117] "RemoveContainer" containerID="8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf" Jul 06 20:54:46.855994 ip-10-0-133-106 kubenswrapper[2566]: E0706 20:54:46.855970 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf\": container with ID starting with 8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf not found: ID does not exist" containerID="8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf" Jul 06 20:54:46.856062 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.856002 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf"} err="failed to get container status \"8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf\": rpc error: code = NotFound desc = could not find container \"8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf\": container with ID starting with 8cfde7c98c89c680a76b02a963ffa90921a0e90e050b2c77145932984ea0fbdf not found: ID does not exist" Jul 06 20:54:46.868433 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.868416 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-model-cache\") pod \"2a9addf2-5426-4def-bae1-d5019f86c95e\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " Jul 06 20:54:46.868513 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.868446 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqnhs\" (UniqueName: \"kubernetes.io/projected/2a9addf2-5426-4def-bae1-d5019f86c95e-kube-api-access-hqnhs\") pod \"2a9addf2-5426-4def-bae1-d5019f86c95e\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " Jul 06 20:54:46.868513 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.868468 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-dshm\") pod \"2a9addf2-5426-4def-bae1-d5019f86c95e\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " Jul 06 20:54:46.868614 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.868520 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-tmp-dir\") pod \"2a9addf2-5426-4def-bae1-d5019f86c95e\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " Jul 06 20:54:46.868614 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.868608 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-home\") pod \"2a9addf2-5426-4def-bae1-d5019f86c95e\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " Jul 06 20:54:46.868703 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.868638 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a9addf2-5426-4def-bae1-d5019f86c95e-tls-certs\") pod \"2a9addf2-5426-4def-bae1-d5019f86c95e\" (UID: \"2a9addf2-5426-4def-bae1-d5019f86c95e\") " Jul 06 20:54:46.868703 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.868659 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-model-cache" (OuterVolumeSpecName: "model-cache") pod "2a9addf2-5426-4def-bae1-d5019f86c95e" (UID: "2a9addf2-5426-4def-bae1-d5019f86c95e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:54:46.868860 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.868836 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-model-cache\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:54:46.868951 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.868874 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2a9addf2-5426-4def-bae1-d5019f86c95e" (UID: "2a9addf2-5426-4def-bae1-d5019f86c95e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:54:46.868951 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.868887 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-home" (OuterVolumeSpecName: "home") pod "2a9addf2-5426-4def-bae1-d5019f86c95e" (UID: "2a9addf2-5426-4def-bae1-d5019f86c95e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:54:46.870522 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.870493 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-dshm" (OuterVolumeSpecName: "dshm") pod "2a9addf2-5426-4def-bae1-d5019f86c95e" (UID: "2a9addf2-5426-4def-bae1-d5019f86c95e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:54:46.870637 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.870574 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a9addf2-5426-4def-bae1-d5019f86c95e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2a9addf2-5426-4def-bae1-d5019f86c95e" (UID: "2a9addf2-5426-4def-bae1-d5019f86c95e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:54:46.870637 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.870597 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a9addf2-5426-4def-bae1-d5019f86c95e-kube-api-access-hqnhs" (OuterVolumeSpecName: "kube-api-access-hqnhs") pod "2a9addf2-5426-4def-bae1-d5019f86c95e" (UID: "2a9addf2-5426-4def-bae1-d5019f86c95e"). InnerVolumeSpecName "kube-api-access-hqnhs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:54:46.969235 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.969200 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqnhs\" (UniqueName: \"kubernetes.io/projected/2a9addf2-5426-4def-bae1-d5019f86c95e-kube-api-access-hqnhs\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:54:46.969235 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.969228 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-dshm\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:54:46.969235 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.969238 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-tmp-dir\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:54:46.969502 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.969249 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a9addf2-5426-4def-bae1-d5019f86c95e-home\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:54:46.969502 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:46.969257 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a9addf2-5426-4def-bae1-d5019f86c95e-tls-certs\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\"" Jul 06 20:54:47.168119 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:47.168089 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986"] Jul 06 20:54:47.173183 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:47.173158 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d729fa/rate-test-model-d729fa-kserve-555c6586b7-45986"] Jul 06 20:54:47.668021 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:54:47.667983 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a9addf2-5426-4def-bae1-d5019f86c95e" path="/var/lib/kubelet/pods/2a9addf2-5426-4def-bae1-d5019f86c95e/volumes" Jul 06 20:55:17.944834 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:17.944797 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-fhvmh"] Jul 06 20:55:27.905578 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:27.905531 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722845-88cxv_641b865d-fa84-469d-a769-7b2b32cf207b/cleanup/0.log" Jul 06 20:55:28.363731 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:28.363702 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q_55669313-69e2-4bb1-b40f-08e723f79c2d/copy-manifests/0.log" Jul 06 20:55:28.378520 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:28.378493 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q_55669313-69e2-4bb1-b40f-08e723f79c2d/manager/0.log" Jul 06 20:55:30.286307 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:30.286280 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rxdxg_76fac1fe-66ba-4da1-a360-a2420fd87e35/manager/0.log" Jul 06 20:55:30.746834 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:30.746808 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-fhvmh_9c1888d6-b8b6-489d-a838-1971f31bb10d/limitador/0.log" Jul 06 20:55:30.868468 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:30.868445 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-74nrb_cfd55d03-32f0-4c13-833f-bf53e029ce80/manager/0.log" Jul 06 20:55:31.819012 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:31.818987 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-c24v6_519cc82d-24cb-4f24-944d-23e2cbed9baf/payload-pre-processing/0.log" Jul 06 20:55:31.932045 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:31.932012 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5l9k8_e4212cd5-2741-45bf-a2bc-e25398deca6b/payload-processing/0.log" Jul 06 20:55:32.048143 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:32.048122 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57bb47759f-dq5zh_faff23fe-0eb2-45d5-a5ea-85f830547761/router/0.log" Jul 06 20:55:32.492165 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:32.492114 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vqmmn_12288c3e-b505-4ef0-aa22-14b9d0f00f5b/main/0.log" Jul 06 20:55:32.835593 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:32.835494 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-k5d4r_0705b2e8-c8d7-43e3-9393-75e6d83d6e4b/main/0.log" Jul 06 20:55:32.950996 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:32.950964 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-fnlv2_9cb12f48-9fd3-4950-9df9-821a501faff3/main/0.log" Jul 06 20:55:40.415806 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:40.415775 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tjmw4_985b8c24-44e1-435e-96c0-e99a601b96c1/global-pull-secret-syncer/0.log" Jul 06 20:55:40.541082 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:40.541053 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-q2nlz_1a44f5a2-9953-4ae9-b0be-769c3f4e0d96/konnectivity-agent/0.log" Jul 06 20:55:40.587663 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:40.587635 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-106.ec2.internal_5b7cb81d798809d5a2a79e2bc119037e/haproxy/0.log" Jul 06 20:55:45.071966 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:45.071935 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rxdxg_76fac1fe-66ba-4da1-a360-a2420fd87e35/manager/0.log" Jul 06 20:55:45.384252 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:45.384168 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-fhvmh_9c1888d6-b8b6-489d-a838-1971f31bb10d/limitador/0.log" Jul 06 20:55:45.450191 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:45.450162 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-74nrb_cfd55d03-32f0-4c13-833f-bf53e029ce80/manager/0.log" Jul 06 20:55:47.161718 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:47.161677 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-r85ct_8823196a-ede8-4dd3-a54c-e059bd003283/cluster-monitoring-operator/0.log" Jul 06 20:55:47.379221 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:47.379195 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2lzrs_afd18897-c080-4905-98b3-17b79190b39b/node-exporter/0.log" Jul 06 20:55:47.420292 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:47.420204 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2lzrs_afd18897-c080-4905-98b3-17b79190b39b/kube-rbac-proxy/0.log" Jul 06 20:55:47.463572 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:47.463524 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2lzrs_afd18897-c080-4905-98b3-17b79190b39b/init-textfile/0.log" Jul 06 20:55:49.576702 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:49.576665 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-zqtxq_81143b00-2314-4ecf-ac5e-fd92e429639c/networking-console-plugin/0.log" Jul 06 20:55:50.253493 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.253458 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/1.log" Jul 06 20:55:50.260526 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.260500 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g6t88_62d6ada4-8960-4e60-a605-9621138b9c8d/console-operator/2.log" Jul 06 20:55:50.316424 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.316393 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2"] Jul 06 20:55:50.316784 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.316770 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a9addf2-5426-4def-bae1-d5019f86c95e" containerName="main" Jul 06 20:55:50.316832 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.316786 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a9addf2-5426-4def-bae1-d5019f86c95e" containerName="main" Jul 06 20:55:50.316866 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.316860 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a9addf2-5426-4def-bae1-d5019f86c95e" containerName="main" Jul 06 20:55:50.319930 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.319908 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.323134 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.323114 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hvwvn\"/\"openshift-service-ca.crt\"" Jul 06 20:55:50.323931 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.323909 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hvwvn\"/\"default-dockercfg-dk4dk\"" Jul 06 20:55:50.323931 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.323918 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hvwvn\"/\"kube-root-ca.crt\"" Jul 06 20:55:50.333090 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.333061 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2"] Jul 06 20:55:50.414680 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.414639 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-proc\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.414862 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.414679 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-lib-modules\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.414862 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.414713 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djvcf\" (UniqueName: \"kubernetes.io/projected/581e3cee-ceb8-48bd-8c61-488ef8ff048c-kube-api-access-djvcf\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.414862 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.414789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-sys\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.414862 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.414813 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-podres\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.515677 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.515540 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-lib-modules\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.515677 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.515646 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djvcf\" (UniqueName: \"kubernetes.io/projected/581e3cee-ceb8-48bd-8c61-488ef8ff048c-kube-api-access-djvcf\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.515864 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.515701 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-sys\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.515864 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.515723 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-lib-modules\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.515864 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.515765 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-sys\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.515864 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.515819 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-podres\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.515990 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.515950 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-proc\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.516022 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.515988 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-podres\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.516055 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.516034 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/581e3cee-ceb8-48bd-8c61-488ef8ff048c-proc\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.524354 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.524319 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djvcf\" (UniqueName: \"kubernetes.io/projected/581e3cee-ceb8-48bd-8c61-488ef8ff048c-kube-api-access-djvcf\") pod \"perf-node-gather-daemonset-8k7x2\" (UID: \"581e3cee-ceb8-48bd-8c61-488ef8ff048c\") " pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.629602 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.629535 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:50.775400 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.775312 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2"] Jul 06 20:55:50.779356 ip-10-0-133-106 kubenswrapper[2566]: W0706 20:55:50.779321 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod581e3cee_ceb8_48bd_8c61_488ef8ff048c.slice/crio-4a34e036fe96a8eda48cf62536150e97aaa5d2e5c972c9026e040a5783f25b58 WatchSource:0}: Error finding container 4a34e036fe96a8eda48cf62536150e97aaa5d2e5c972c9026e040a5783f25b58: Status 404 returned error can't find the container with id 4a34e036fe96a8eda48cf62536150e97aaa5d2e5c972c9026e040a5783f25b58 Jul 06 20:55:50.932583 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:50.932540 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-g9qjv_0f0c847d-119c-469b-943d-e385d909b9b3/download-server/0.log" Jul 06 20:55:51.151600 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:51.151484 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" event={"ID":"581e3cee-ceb8-48bd-8c61-488ef8ff048c","Type":"ContainerStarted","Data":"854c55dac69f4af0363a773f2469627347f1a093c52edbd58e1b4398a67a1312"} Jul 06 20:55:51.151600 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:51.151540 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" event={"ID":"581e3cee-ceb8-48bd-8c61-488ef8ff048c","Type":"ContainerStarted","Data":"4a34e036fe96a8eda48cf62536150e97aaa5d2e5c972c9026e040a5783f25b58"} Jul 06 20:55:51.151600 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:51.151588 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:51.170968 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:51.170918 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" podStartSLOduration=1.170904101 podStartE2EDuration="1.170904101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:55:51.169838938 +0000 UTC m=+2488.015703667" watchObservedRunningTime="2026-07-06 20:55:51.170904101 +0000 UTC m=+2488.016768805" Jul 06 20:55:51.581772 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:51.581743 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-r6k8b_c582958d-acac-453d-8a5b-eb998905e4b0/volume-data-source-validator/0.log" Jul 06 20:55:52.653543 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:52.653496 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x27pv_e23f0332-a4f0-4638-8ac3-a9d89fa3564a/dns/0.log" Jul 06 20:55:52.678682 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:52.678654 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x27pv_e23f0332-a4f0-4638-8ac3-a9d89fa3564a/kube-rbac-proxy/0.log" Jul 06 20:55:52.704362 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:52.704332 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6q7sk_5d7c76e4-8c48-47ca-9514-6d26ac324460/dns-node-resolver/0.log" Jul 06 20:55:53.359116 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:53.359091 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tlhdp_1c6ffb1c-9153-4f2d-83ba-967ed6a026a1/node-ca/0.log" Jul 06 20:55:54.662708 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:54.662677 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-c24v6_519cc82d-24cb-4f24-944d-23e2cbed9baf/payload-pre-processing/0.log" Jul 06 20:55:54.716723 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:54.716685 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5l9k8_e4212cd5-2741-45bf-a2bc-e25398deca6b/payload-processing/0.log" Jul 06 20:55:54.740056 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:54.740030 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57bb47759f-dq5zh_faff23fe-0eb2-45d5-a5ea-85f830547761/router/0.log" Jul 06 20:55:55.405862 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:55.405828 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ckhsf_52e1eff4-f7e7-4307-a72b-2bd3b2e58241/serve-healthcheck-canary/0.log" Jul 06 20:55:55.911111 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:55.911079 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-pwjb2_39246747-c52b-40a4-b3d5-bf8fed1223dd/insights-operator/0.log" Jul 06 20:55:56.088387 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:56.088356 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lvtdc_748c7f38-4aa4-4279-b7db-b0eb53d4e9e7/kube-rbac-proxy/0.log" Jul 06 20:55:56.111965 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:56.111932 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lvtdc_748c7f38-4aa4-4279-b7db-b0eb53d4e9e7/exporter/0.log" Jul 06 20:55:56.134826 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:56.134798 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lvtdc_748c7f38-4aa4-4279-b7db-b0eb53d4e9e7/extractor/0.log" Jul 06 20:55:57.165947 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:57.165921 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hvwvn/perf-node-gather-daemonset-8k7x2" Jul 06 20:55:58.512095 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:58.512069 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722845-88cxv_641b865d-fa84-469d-a769-7b2b32cf207b/cleanup/0.log" Jul 06 20:55:58.836423 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:58.836393 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q_55669313-69e2-4bb1-b40f-08e723f79c2d/manager/0.log" Jul 06 20:55:58.858854 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:55:58.858831 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b6b7b7bb-fdg6q_55669313-69e2-4bb1-b40f-08e723f79c2d/copy-manifests/0.log" Jul 06 20:56:00.513362 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:00.513339 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-86b486bf74-r8brg_e2a0df31-c44c-4baa-aad1-2ea07496d89a/manager/0.log" Jul 06 20:56:00.870053 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:00.870023 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-c24v6_519cc82d-24cb-4f24-944d-23e2cbed9baf/payload-pre-processing/0.log" Jul 06 20:56:00.904867 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:00.904842 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5l9k8_e4212cd5-2741-45bf-a2bc-e25398deca6b/payload-processing/0.log" Jul 06 20:56:00.920763 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:00.920747 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57bb47759f-dq5zh_faff23fe-0eb2-45d5-a5ea-85f830547761/router/0.log" Jul 06 20:56:02.383118 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:02.383047 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-c24v6_519cc82d-24cb-4f24-944d-23e2cbed9baf/payload-pre-processing/0.log" Jul 06 20:56:02.419710 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:02.419683 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5l9k8_e4212cd5-2741-45bf-a2bc-e25398deca6b/payload-processing/0.log" Jul 06 20:56:02.435015 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:02.434992 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57bb47759f-dq5zh_faff23fe-0eb2-45d5-a5ea-85f830547761/router/0.log" Jul 06 20:56:05.399935 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:05.399906 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-snzgw_900867c2-1a0f-448d-9909-5c0cf59a161a/migrator/0.log" Jul 06 20:56:05.422910 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:05.422885 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-snzgw_900867c2-1a0f-448d-9909-5c0cf59a161a/graceful-termination/0.log" Jul 06 20:56:05.791474 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:05.791441 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-w6b76_5dd21d42-e317-4776-9822-2dbb049973f2/kube-storage-version-migrator-operator/1.log" Jul 06 20:56:05.791904 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:05.791886 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-w6b76_5dd21d42-e317-4776-9822-2dbb049973f2/kube-storage-version-migrator-operator/0.log" Jul 06 20:56:06.819184 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:06.819158 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hb74l_12b0e04f-1230-4820-a7ce-93a5ce83ef68/kube-multus-additional-cni-plugins/0.log" Jul 06 20:56:06.847375 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:06.847350 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hb74l_12b0e04f-1230-4820-a7ce-93a5ce83ef68/egress-router-binary-copy/0.log" Jul 06 20:56:06.872331 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:06.872301 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hb74l_12b0e04f-1230-4820-a7ce-93a5ce83ef68/cni-plugins/0.log" Jul 06 20:56:06.896965 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:06.896941 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hb74l_12b0e04f-1230-4820-a7ce-93a5ce83ef68/bond-cni-plugin/0.log" Jul 06 20:56:06.919285 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:06.919263 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hb74l_12b0e04f-1230-4820-a7ce-93a5ce83ef68/routeoverride-cni/0.log" Jul 06 20:56:06.946350 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:06.946331 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hb74l_12b0e04f-1230-4820-a7ce-93a5ce83ef68/whereabouts-cni-bincopy/0.log" Jul 06 20:56:06.971981 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:06.971965 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hb74l_12b0e04f-1230-4820-a7ce-93a5ce83ef68/whereabouts-cni/0.log" Jul 06 20:56:07.386891 ip-10-0-133-106 kubenswrapper[2566]: I0706 20:56:07.386860 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q55sw_19dcbafe-60f4-40a5-9ff5-a5c7b48f3ce8/kube-multus/0.log"