Jul 06 14:26:55.690354 ip-10-0-137-102 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 14:26:55.690369 ip-10-0-137-102 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 14:26:55.690379 ip-10-0-137-102 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 14:26:55.690697 ip-10-0-137-102 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 14:27:05.939369 ip-10-0-137-102 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 14:27:05.939386 ip-10-0-137-102 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 13e859194d61476faa2402e2f2abadc5 -- Jul 06 14:28:52.241463 ip-10-0-137-102 systemd[1]: Starting Kubernetes Kubelet... Jul 06 14:28:53.037983 ip-10-0-137-102 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:28:53.037983 ip-10-0-137-102 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 14:28:53.037983 ip-10-0-137-102 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:28:53.037983 ip-10-0-137-102 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 14:28:53.037983 ip-10-0-137-102 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:28:53.039534 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.039442 2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 14:28:53.044574 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044544 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:28:53.044574 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044570 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044577 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044588 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044594 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044601 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044606 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044612 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044616 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044621 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044626 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044631 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044636 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044640 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044646 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044660 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044665 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044670 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044675 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044680 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044685 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:28:53.044731 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044689 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044694 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044699 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044703 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044708 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044712 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044721 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044724 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044727 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044730 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044734 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044737 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044744 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044748 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044751 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044755 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044758 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044950 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044956 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044959 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:28:53.045427 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044963 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044967 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044971 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044974 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044977 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044985 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044989 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044993 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.044996 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045000 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045003 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045006 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045009 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045013 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045016 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045019 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045027 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045031 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045034 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:28:53.045946 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045037 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045040 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045042 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045047 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045050 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045054 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045057 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045060 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045063 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045067 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045070 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045073 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045076 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045079 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045082 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045088 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045092 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045095 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045098 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:28:53.046435 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045101 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045104 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045107 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045110 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045114 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045117 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045120 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045122 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045125 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045129 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045131 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045134 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045138 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045141 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045144 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045147 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045149 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045152 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045155 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045158 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:28:53.046918 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045932 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045940 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045945 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045949 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045952 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045956 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045959 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045962 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045964 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.045998 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046003 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046006 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046010 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046013 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046015 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046018 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046021 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046025 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046028 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046031 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:28:53.047484 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046033 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046036 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046039 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046042 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046045 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046047 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046051 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046053 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046057 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046060 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046062 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046065 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046068 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046070 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046075 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046079 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046081 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046084 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046087 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046090 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:28:53.047995 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046093 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046095 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046099 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046102 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046104 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046107 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046110 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046113 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046115 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046118 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046121 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046124 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046127 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046130 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046132 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046136 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046140 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046143 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046146 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046149 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:28:53.048536 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046152 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046155 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046158 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046160 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046163 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046166 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046169 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046172 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046175 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046178 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046182 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046185 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046188 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046190 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046193 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046196 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046198 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046201 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046204 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:28:53.049039 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046208 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046212 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046215 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046218 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046221 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046224 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046227 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046229 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046232 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046235 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046238 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046241 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046244 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046246 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046249 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046253 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046255 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046258 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046260 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046263 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:28:53.049576 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046687 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046693 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046696 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046699 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046703 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046706 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046709 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046712 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046715 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046718 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046721 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046724 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046727 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046729 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046732 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046735 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046738 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046742 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046745 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046748 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:28:53.050098 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046750 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046753 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046756 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046759 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046762 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046765 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046768 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046771 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046773 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046776 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046779 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046781 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046785 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046788 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046791 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046794 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046796 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046799 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046802 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046805 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:28:53.050641 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046808 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046810 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046813 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046816 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046819 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046821 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046824 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046827 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046830 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046838 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046841 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046844 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046847 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046852 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046856 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046859 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046862 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046864 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046867 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046870 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:28:53.051231 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046873 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046876 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046879 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046882 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046884 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046889 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046893 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046896 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046899 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046902 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046905 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046908 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046911 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046916 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046919 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046923 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046926 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046929 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046932 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:28:53.051782 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046936 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046939 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046943 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046947 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046949 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046952 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046955 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046957 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046960 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046963 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046965 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046968 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046971 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046973 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046976 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046979 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046982 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046985 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046987 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.046990 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:28:53.052270 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047068 2569 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047076 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047082 2569 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047086 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047091 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047095 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047100 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047104 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047107 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047110 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047114 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047118 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047121 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047125 2569 flags.go:64] FLAG: --cgroup-root="" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047130 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047133 2569 flags.go:64] FLAG: --client-ca-file="" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047136 2569 flags.go:64] FLAG: --cloud-provider="external" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047140 2569 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047145 2569 flags.go:64] FLAG: --cluster-domain="" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047148 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047151 2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047154 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047158 2569 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047162 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 14:28:53.052838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047166 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047169 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047172 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047176 2569 flags.go:64] FLAG: --contention-profiling="false" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047179 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047182 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047185 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047189 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047193 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047196 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047199 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047205 2569 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047208 2569 flags.go:64] FLAG: --enable-server="true" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047212 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047216 2569 flags.go:64] FLAG: --event-burst="100" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047220 2569 flags.go:64] FLAG: --event-qps="50" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047223 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047226 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047229 2569 flags.go:64] FLAG: --eviction-hard="" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047234 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047237 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047240 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047244 2569 flags.go:64] FLAG: --eviction-soft="" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047247 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047250 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 14:28:53.053512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047254 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047258 2569 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047261 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047264 2569 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047267 2569 flags.go:64] FLAG: --feature-gates="" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047301 2569 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047307 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047312 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047315 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047319 2569 flags.go:64] FLAG: --healthz-port="10248" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047323 2569 flags.go:64] FLAG: --help="false" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047326 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047329 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047332 2569 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047335 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047339 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047343 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047346 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047350 2569 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047353 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047357 2569 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047360 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047363 2569 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047366 2569 flags.go:64] FLAG: --kube-reserved="" Jul 06 14:28:53.054191 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047369 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047372 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047375 2569 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047378 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047381 2569 flags.go:64] FLAG: --lock-file="" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047385 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047389 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047392 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047398 2569 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047401 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047404 2569 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047407 2569 flags.go:64] FLAG: --logging-format="text" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047411 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047414 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047417 2569 flags.go:64] FLAG: --manifest-url="" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047420 2569 flags.go:64] FLAG: --manifest-url-header="" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047424 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047428 2569 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047432 2569 flags.go:64] FLAG: --max-pods="110" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047435 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047438 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047441 2569 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047444 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047448 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047451 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 14:28:53.054808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047454 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047463 2569 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047466 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047469 2569 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047473 2569 flags.go:64] FLAG: --pod-cidr="" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047476 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047482 2569 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047486 2569 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047490 2569 flags.go:64] FLAG: --pods-per-core="0" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047493 2569 flags.go:64] FLAG: --port="10250" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047496 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047499 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e1e81271da5b0845" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047503 2569 flags.go:64] FLAG: --qos-reserved="" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047507 2569 flags.go:64] FLAG: --read-only-port="10255" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047510 2569 flags.go:64] FLAG: --register-node="true" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047513 2569 flags.go:64] FLAG: --register-with-taints="" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047517 2569 flags.go:64] FLAG: --registry-burst="10" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047521 2569 flags.go:64] FLAG: --registry-qps="5" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047524 2569 flags.go:64] FLAG: --reserved-cpus="" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047527 2569 flags.go:64] FLAG: --reserved-memory="" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047531 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047534 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047537 2569 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047540 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047543 2569 flags.go:64] FLAG: --runonce="false" Jul 06 14:28:53.055472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047546 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047549 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047552 2569 flags.go:64] FLAG: --seccomp-default="false" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047555 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047558 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047562 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047565 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047568 2569 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047573 2569 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047576 2569 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047579 2569 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047582 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047585 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047588 2569 flags.go:64] FLAG: --system-cgroups="" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047591 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047597 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047600 2569 flags.go:64] FLAG: --tls-cert-file="" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047603 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047608 2569 flags.go:64] FLAG: --tls-min-version="" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047611 2569 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047615 2569 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047618 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047621 2569 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047625 2569 flags.go:64] FLAG: --v="2" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047629 2569 flags.go:64] FLAG: --version="false" Jul 06 14:28:53.056120 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047634 2569 flags.go:64] FLAG: --vmodule="" Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047638 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.047641 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047795 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047799 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047803 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047806 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047809 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047812 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047815 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047818 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047821 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047824 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047827 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047830 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047837 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047839 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047842 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047845 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047848 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:28:53.056830 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047851 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047854 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047857 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047860 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047863 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047866 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047868 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047873 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047875 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047878 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047881 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047884 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047887 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047890 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047894 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047896 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047899 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047902 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047905 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:28:53.057395 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047908 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047910 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047913 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047916 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047919 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047922 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047925 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047928 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047932 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047935 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047938 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047940 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047943 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047946 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047949 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047951 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047954 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047957 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047960 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047962 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:28:53.057890 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047966 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047969 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047972 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047975 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047977 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047980 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047983 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047988 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047992 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.047996 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048000 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048003 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048006 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048011 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048014 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048017 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048020 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048024 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048026 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:28:53.058410 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048030 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048034 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048037 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048040 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048043 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048046 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048048 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048051 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048054 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048057 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048060 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048062 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048065 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048068 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048071 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048074 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048077 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048080 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048083 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048095 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:28:53.058902 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048098 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048102 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048105 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.048108 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.048113 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.055529 2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.055544 2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.055563 2569 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.055569 2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055630 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055635 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055638 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055641 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055644 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:28:53.059432 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055647 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055651 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055654 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055657 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055660 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055663 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055666 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055669 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055672 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055675 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055678 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055681 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055683 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055686 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055689 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055692 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055695 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055698 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055702 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:28:53.059838 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055712 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055716 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055719 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055722 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055725 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055729 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055732 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055735 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055738 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055741 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055744 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055746 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055749 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055752 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055755 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055758 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055760 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055763 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055766 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055769 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:28:53.060363 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055771 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055774 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055777 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055780 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055783 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055785 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055788 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055791 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055793 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055797 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055799 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055802 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055805 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055811 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055814 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055817 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055819 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055823 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055826 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055829 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:28:53.060882 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055831 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055834 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055837 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055840 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055842 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055846 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055849 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055851 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055854 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055857 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055860 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055863 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055865 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055868 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055871 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055874 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055877 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055880 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055884 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055887 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:28:53.061498 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055890 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055893 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055896 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055899 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055901 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055904 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055908 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055910 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055913 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055916 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055921 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055924 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055928 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055931 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:28:53.062002 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.055934 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.055939 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056057 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056062 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056065 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056068 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056072 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056075 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056078 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056081 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056085 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056088 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056091 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056094 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:28:53.062419 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056097 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056100 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056103 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056106 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056109 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056111 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056114 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056117 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056119 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056122 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056125 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056129 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056132 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056135 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056138 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056141 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056144 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056147 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056149 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056152 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:28:53.062786 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056154 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056157 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056160 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056163 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056166 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056169 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056171 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056174 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056177 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056180 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056182 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056185 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056187 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056192 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056196 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056199 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056202 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056204 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056208 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:28:53.063309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056211 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056214 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056216 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056219 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056222 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056225 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056228 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056231 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056234 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056236 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056239 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056242 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056245 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056247 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056250 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056253 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056255 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056258 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056261 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056263 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:28:53.063789 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056266 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056269 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056272 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056274 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056277 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056281 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056301 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056305 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056308 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056311 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056314 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056316 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056319 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056322 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056326 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056330 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056332 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056335 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056339 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:28:53.064310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056342 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056345 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056348 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056351 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056353 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056356 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056359 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056361 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:53.056364 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.056370 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.056469 2569 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.059656 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.061220 2569 server.go:1019] "Starting client certificate rotation" Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.061336 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 14:28:53.064807 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.062199 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 14:28:53.065165 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.064059 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 14:28:53.069237 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.069218 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 14:28:53.079090 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.079067 2569 log.go:25] "Validated CRI v1 runtime API" Jul 06 14:28:53.081875 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.081851 2569 log.go:25] "Validated CRI v1 image API" Jul 06 14:28:53.083603 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.083588 2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 14:28:53.087529 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.087509 2569 fs.go:135] Filesystem UUIDs: map[168e5c14-0a7d-499d-b98d-9b596789f1d4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f5e356e3-d2a0-42dd-8559-2f593c94b16d:/dev/nvme0n1p4] Jul 06 14:28:53.087601 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.087528 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 14:28:53.094054 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.093936 2569 manager.go:216] Machine: {Timestamp:2026-07-06 14:28:53.092504475 +0000 UTC m=+0.733474407 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099232 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec236ddc3202022e3aa0add1a9758d3b SystemUUID:ec236ddc-3202-022e-3aa0-add1a9758d3b BootID:13e85919-4d61-476f-aa24-02e2f2abadc5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d1:65:4e:74:53 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d1:65:4e:74:53 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:a0:12:14:5c:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 14:28:53.094054 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.094047 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 14:28:53.094177 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.094150 2569 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 14:28:53.097350 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.097319 2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 14:28:53.097543 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.097356 2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-102.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 14:28:53.098116 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.098103 2569 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 14:28:53.098185 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.098121 2569 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 14:28:53.098185 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.098132 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 14:28:53.098913 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.098899 2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 14:28:53.100497 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.100483 2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 14:28:53.103253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.103235 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 14:28:53.104479 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.104464 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 06 14:28:53.104625 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.104613 2569 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 14:28:53.104699 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.104653 2569 kubelet.go:486] "Attempting to sync node with API server" Jul 06 14:28:53.104699 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.104677 2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 14:28:53.104699 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.104695 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 14:28:53.104873 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.104709 2569 kubelet.go:398] "Adding apiserver pod source" Jul 06 14:28:53.104873 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.104725 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 14:28:53.105127 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.105112 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 14:28:53.105192 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.105137 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 14:28:53.108074 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.108057 2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 14:28:53.110068 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.110054 2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 14:28:53.110128 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.110087 2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 14:28:53.111127 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111111 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 14:28:53.111214 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111143 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 14:28:53.111214 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111154 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 14:28:53.111214 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111161 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 14:28:53.111214 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111175 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 14:28:53.111214 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111188 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 14:28:53.111214 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111204 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 14:28:53.111214 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111212 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 14:28:53.111443 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111221 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 14:28:53.111443 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111259 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 14:28:53.111443 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111291 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 14:28:53.111443 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.111300 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 14:28:53.112093 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.112080 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 14:28:53.112132 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.112095 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 14:28:53.115558 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.115542 2569 server.go:1268] "Started kubelet" Jul 06 14:28:53.115639 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.115604 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 14:28:53.115707 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.115630 2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 14:28:53.115841 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.115819 2569 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 14:28:53.116663 ip-10-0-137-102 systemd[1]: Started Kubernetes Kubelet. Jul 06 14:28:53.117311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.117102 2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 14:28:53.117378 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.117286 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 14:28:53.117515 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.117393 2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-102.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 14:28:53.118050 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.118024 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 14:28:53.118832 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.118812 2569 server.go:310] "Adding debug handlers to kubelet server" Jul 06 14:28:53.122112 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.122084 2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 14:28:53.123188 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.122240 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-102.ec2.internal.18bfb9adb134bf53 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-102.ec2.internal,UID:ip-10-0-137-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-102.ec2.internal,},FirstTimestamp:2026-07-06 14:28:53.115510611 +0000 UTC m=+0.756480536,LastTimestamp:2026-07-06 14:28:53.115510611 +0000 UTC m=+0.756480536,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-102.ec2.internal,}" Jul 06 14:28:53.124375 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.124356 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 14:28:53.124449 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.124361 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 14:28:53.124449 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.124440 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:53.125076 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125055 2569 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 14:28:53.125237 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125087 2569 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 14:28:53.125237 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125059 2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 14:28:53.125237 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125201 2569 reconstruct.go:97] "Volume reconstruction finished" Jul 06 14:28:53.125237 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125218 2569 reconciler.go:29] "Reconciler: start to sync state" Jul 06 14:28:53.125441 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125308 2569 factory.go:55] Registering systemd factory Jul 06 14:28:53.125441 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125326 2569 factory.go:223] Registration of the systemd container factory successfully Jul 06 14:28:53.125557 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125543 2569 factory.go:153] Registering CRI-O factory Jul 06 14:28:53.125609 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125560 2569 factory.go:223] Registration of the crio container factory successfully Jul 06 14:28:53.125667 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125652 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 14:28:53.125719 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125680 2569 factory.go:103] Registering Raw factory Jul 06 14:28:53.125719 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.125698 2569 manager.go:1196] Started watching for new ooms in manager Jul 06 14:28:53.126128 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.126110 2569 manager.go:349] Starting recovery of all containers Jul 06 14:28:53.128522 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.128498 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 14:28:53.128598 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.128534 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 14:28:53.139382 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.139365 2569 manager.go:354] Recovery completed Jul 06 14:28:53.143578 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.143564 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:28:53.145569 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.145548 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:28:53.145642 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.145584 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:28:53.145642 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.145598 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:28:53.146173 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.146156 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 14:28:53.146173 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.146172 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 14:28:53.146271 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.146190 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 06 14:28:53.146271 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.146191 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:28:53.147033 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.147016 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:28:53.147119 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.147041 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:28:53.147119 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.147056 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:28:53.148549 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.148531 2569 policy_none.go:49] "None policy: Start" Jul 06 14:28:53.148632 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.148553 2569 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 14:28:53.148632 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.148570 2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 14:28:53.148717 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.148581 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-102.ec2.internal.18bfb9adb2ff6519 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-102.ec2.internal,UID:ip-10-0-137-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-102.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-102.ec2.internal,},FirstTimestamp:2026-07-06 14:28:53.145568537 +0000 UTC m=+0.786538460,LastTimestamp:2026-07-06 14:28:53.145568537 +0000 UTC m=+0.786538460,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-102.ec2.internal,}" Jul 06 14:28:53.150678 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.150662 2569 policy_none.go:47] "Start" Jul 06 14:28:53.159032 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.158963 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-102.ec2.internal.18bfb9adb2ffc2ca default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-102.ec2.internal,UID:ip-10-0-137-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-102.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-102.ec2.internal,},FirstTimestamp:2026-07-06 14:28:53.145592522 +0000 UTC m=+0.786562446,LastTimestamp:2026-07-06 14:28:53.145592522 +0000 UTC m=+0.786562446,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-102.ec2.internal,}" Jul 06 14:28:53.161266 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.161210 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-102.ec2.internal.18bfb9adb2ffec18 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-102.ec2.internal,UID:ip-10-0-137-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-102.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-102.ec2.internal,},FirstTimestamp:2026-07-06 14:28:53.145603096 +0000 UTC m=+0.786573020,LastTimestamp:2026-07-06 14:28:53.145603096 +0000 UTC m=+0.786573020,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-102.ec2.internal,}" Jul 06 14:28:53.172155 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.172070 2569 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-102.ec2.internal.18bfb9adb2ff6519\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-102.ec2.internal.18bfb9adb2ff6519 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-102.ec2.internal,UID:ip-10-0-137-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-102.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-102.ec2.internal,},FirstTimestamp:2026-07-06 14:28:53.145568537 +0000 UTC m=+0.786538460,LastTimestamp:2026-07-06 14:28:53.147029263 +0000 UTC m=+0.787999187,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-102.ec2.internal,}" Jul 06 14:28:53.174965 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.174869 2569 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-102.ec2.internal.18bfb9adb2ffc2ca\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-102.ec2.internal.18bfb9adb2ffc2ca default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-102.ec2.internal,UID:ip-10-0-137-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-102.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-102.ec2.internal,},FirstTimestamp:2026-07-06 14:28:53.145592522 +0000 UTC m=+0.786562446,LastTimestamp:2026-07-06 14:28:53.147048922 +0000 UTC m=+0.788018852,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-102.ec2.internal,}" Jul 06 14:28:53.184054 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.183989 2569 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-102.ec2.internal.18bfb9adb2ffec18\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-102.ec2.internal.18bfb9adb2ffec18 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-102.ec2.internal,UID:ip-10-0-137-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-102.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-102.ec2.internal,},FirstTimestamp:2026-07-06 14:28:53.145603096 +0000 UTC m=+0.786573020,LastTimestamp:2026-07-06 14:28:53.147060805 +0000 UTC m=+0.788030729,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-102.ec2.internal,}" Jul 06 14:28:53.188848 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.188832 2569 manager.go:337] "Starting Device Plugin manager" Jul 06 14:28:53.188947 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.188930 2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 14:28:53.189011 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.188951 2569 server.go:87] "Starting device plugin registration server" Jul 06 14:28:53.189201 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.189187 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 14:28:53.189262 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.189205 2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 14:28:53.189341 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.189313 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 14:28:53.189415 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.189396 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 14:28:53.189415 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.189412 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 14:28:53.189936 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.189916 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 14:28:53.190011 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.189966 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:53.204672 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.204612 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-102.ec2.internal.18bfb9adb5b55f7f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-102.ec2.internal,UID:ip-10-0-137-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-137-102.ec2.internal,},FirstTimestamp:2026-07-06 14:28:53.191049087 +0000 UTC m=+0.832019003,LastTimestamp:2026-07-06 14:28:53.191049087 +0000 UTC m=+0.832019003,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-102.ec2.internal,}" Jul 06 14:28:53.229421 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.229382 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 14:28:53.239087 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.230560 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 14:28:53.239087 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.230593 2569 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 14:28:53.239087 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.230613 2569 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 14:28:53.239087 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.230642 2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 14:28:53.243560 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.243534 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Jul 06 14:28:53.275840 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.275816 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5jdlz" Jul 06 14:28:53.286517 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.286491 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5jdlz" Jul 06 14:28:53.290103 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.290067 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:28:53.290910 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.290893 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:28:53.290986 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.290926 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:28:53.290986 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.290940 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:28:53.290986 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.290962 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.310449 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.310429 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.310534 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.310450 2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-102.ec2.internal\": node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:53.331216 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.331193 2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-102.ec2.internal"] Jul 06 14:28:53.331286 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.331263 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:28:53.332812 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.332788 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:28:53.332916 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.332818 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:28:53.332916 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.332828 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:28:53.335131 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.335117 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:28:53.335262 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.335246 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.335322 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.335278 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:28:53.335816 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.335798 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:28:53.335816 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.335825 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:28:53.335953 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.335835 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:28:53.335953 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.335800 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:28:53.335953 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.335905 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:28:53.335953 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.335922 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:28:53.338085 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.338067 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.338174 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.338098 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:28:53.338747 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.338734 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:28:53.338806 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.338763 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:28:53.338806 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.338779 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:28:53.353852 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.353833 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-102.ec2.internal\" not found" node="ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.357477 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.357462 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-102.ec2.internal\" not found" node="ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.416240 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.416204 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:53.516511 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.516481 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:53.526902 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.526882 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecbb59684f17544ae54fb17e71a28592-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal\" (UID: \"ecbb59684f17544ae54fb17e71a28592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.526995 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.526911 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61c1ee201a8907e80dca74652a24fb31-config\") pod \"kube-apiserver-proxy-ip-10-0-137-102.ec2.internal\" (UID: \"61c1ee201a8907e80dca74652a24fb31\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.526995 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.526928 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ecbb59684f17544ae54fb17e71a28592-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal\" (UID: \"ecbb59684f17544ae54fb17e71a28592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.617577 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.617499 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:53.627637 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.627606 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecbb59684f17544ae54fb17e71a28592-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal\" (UID: \"ecbb59684f17544ae54fb17e71a28592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.627736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.627642 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61c1ee201a8907e80dca74652a24fb31-config\") pod \"kube-apiserver-proxy-ip-10-0-137-102.ec2.internal\" (UID: \"61c1ee201a8907e80dca74652a24fb31\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.627736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.627662 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ecbb59684f17544ae54fb17e71a28592-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal\" (UID: \"ecbb59684f17544ae54fb17e71a28592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.627736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.627701 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ecbb59684f17544ae54fb17e71a28592-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal\" (UID: \"ecbb59684f17544ae54fb17e71a28592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.627736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.627729 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61c1ee201a8907e80dca74652a24fb31-config\") pod \"kube-apiserver-proxy-ip-10-0-137-102.ec2.internal\" (UID: \"61c1ee201a8907e80dca74652a24fb31\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.627893 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.627735 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecbb59684f17544ae54fb17e71a28592-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal\" (UID: \"ecbb59684f17544ae54fb17e71a28592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.658022 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.657997 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.660358 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.660343 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-102.ec2.internal" Jul 06 14:28:53.717736 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.717697 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:53.817799 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.817772 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:53.917948 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:53.917870 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:53.966074 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:53.966052 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:28:54.018984 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:54.018948 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:54.060140 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.060104 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 14:28:54.060778 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.060275 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 14:28:54.084009 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.083989 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:28:54.119157 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:54.119138 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:54.171529 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.171472 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:28:54.172432 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.172410 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:28:54.172542 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.172449 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:28:54.172542 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.172466 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:28:54.219826 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:54.219790 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:54.291460 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.291420 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 14:23:54 +0000 UTC" deadline="2027-12-24 22:10:39.867864146 +0000 UTC" Jul 06 14:28:54.291460 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.291456 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12871h41m45.576412468s" Jul 06 14:28:54.320643 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:54.320608 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:54.393270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:54.393227 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c1ee201a8907e80dca74652a24fb31.slice/crio-ed542b0dda8209194a80344d6505385d0b0530660065a2fac2ae35bd8f15acb3 WatchSource:0}: Error finding container ed542b0dda8209194a80344d6505385d0b0530660065a2fac2ae35bd8f15acb3: Status 404 returned error can't find the container with id ed542b0dda8209194a80344d6505385d0b0530660065a2fac2ae35bd8f15acb3 Jul 06 14:28:54.393766 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:54.393746 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecbb59684f17544ae54fb17e71a28592.slice/crio-824e91260244153e77d494e37351c3536a5fbf092d3c649ed0d092904f41ed46 WatchSource:0}: Error finding container 824e91260244153e77d494e37351c3536a5fbf092d3c649ed0d092904f41ed46: Status 404 returned error can't find the container with id 824e91260244153e77d494e37351c3536a5fbf092d3c649ed0d092904f41ed46 Jul 06 14:28:54.398744 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.398726 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:28:54.421048 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:54.421023 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:54.528069 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:54.527992 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-102.ec2.internal\" not found" Jul 06 14:28:54.558852 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.558831 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:28:54.625604 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.625571 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-102.ec2.internal" Jul 06 14:28:54.637045 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.637025 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 14:28:54.637913 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.637900 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" Jul 06 14:28:54.656012 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.655996 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 14:28:54.765500 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:54.765474 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:28:55.106720 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.106631 2569 apiserver.go:52] "Watching apiserver" Jul 06 14:28:55.120403 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.120376 2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 14:28:55.120863 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.120826 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-dzg6l","kube-system/kube-apiserver-proxy-ip-10-0-137-102.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl","openshift-cluster-node-tuning-operator/tuned-bzqcw","openshift-dns/node-resolver-vvm5c","openshift-image-registry/node-ca-99szs","openshift-multus/multus-vsjzf","openshift-multus/network-metrics-daemon-wdpvc","kube-system/konnectivity-agent-75l8n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal","openshift-multus/multus-additional-cni-plugins-bmxf6","openshift-network-diagnostics/network-check-target-hc5qx","openshift-network-operator/iptables-alerter-bz7tn","openshift-ovn-kubernetes/ovnkube-node-q7ks8"] Jul 06 14:28:55.123933 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.123908 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.124618 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.124597 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 14:28:55.126064 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.126039 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bz7tn" Jul 06 14:28:55.128218 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.128196 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 14:28:55.128336 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.128230 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 14:28:55.128529 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.128512 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.130844 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.130822 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.131607 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.131414 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 14:28:55.132220 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.132199 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:28:55.132323 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.132263 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q99x9\"" Jul 06 14:28:55.132479 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.132464 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7pxb5\"" Jul 06 14:28:55.132546 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.132530 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 14:28:55.132729 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.132640 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 14:28:55.132729 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.132712 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 14:28:55.132815 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.132784 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 14:28:55.132903 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.132886 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 14:28:55.133130 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.133112 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hh8bs\"" Jul 06 14:28:55.133367 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.133347 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 14:28:55.134491 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.134174 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vvm5c" Jul 06 14:28:55.134846 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.134705 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 14:28:55.134846 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.134731 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:28:55.135766 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.135556 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-99szs" Jul 06 14:28:55.136345 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136319 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-var-lib-cni-bin\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136443 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136362 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-var-lib-cni-multus\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136443 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136405 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-system-cni-dir\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136443 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136433 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-multus-cni-dir\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136616 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136464 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-cnibin\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136616 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136489 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-os-release\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136616 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136537 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-multus-conf-dir\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136616 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136577 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45eec7f4-47b3-4a95-920c-ef8cc0a710c2-iptables-alerter-script\") pod \"iptables-alerter-bz7tn\" (UID: \"45eec7f4-47b3-4a95-920c-ef8cc0a710c2\") " pod="openshift-network-operator/iptables-alerter-bz7tn" Jul 06 14:28:55.136616 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136604 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-multus-socket-dir-parent\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136882 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136639 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-run-k8s-cni-cncf-io\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136882 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136663 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-var-lib-kubelet\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136882 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136725 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-hostroot\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136882 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136782 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d071527c-3279-43ea-8b14-239560c6dc6e-multus-daemon-config\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.136882 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136808 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45eec7f4-47b3-4a95-920c-ef8cc0a710c2-host-slash\") pod \"iptables-alerter-bz7tn\" (UID: \"45eec7f4-47b3-4a95-920c-ef8cc0a710c2\") " pod="openshift-network-operator/iptables-alerter-bz7tn" Jul 06 14:28:55.136882 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136836 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7sv9\" (UniqueName: \"kubernetes.io/projected/45eec7f4-47b3-4a95-920c-ef8cc0a710c2-kube-api-access-h7sv9\") pod \"iptables-alerter-bz7tn\" (UID: \"45eec7f4-47b3-4a95-920c-ef8cc0a710c2\") " pod="openshift-network-operator/iptables-alerter-bz7tn" Jul 06 14:28:55.136882 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136864 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d071527c-3279-43ea-8b14-239560c6dc6e-cni-binary-copy\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.137245 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136925 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-run-netns\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.137245 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136953 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2j67\" (UniqueName: \"kubernetes.io/projected/d071527c-3279-43ea-8b14-239560c6dc6e-kube-api-access-z2j67\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.137245 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136975 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-run-multus-certs\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.137245 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.136998 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-etc-kubernetes\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.137480 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.137285 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nvf5d\"" Jul 06 14:28:55.138053 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.138032 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:28:55.139046 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.139027 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 14:28:55.139279 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.139256 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m87tn\"" Jul 06 14:28:55.139541 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.139520 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 14:28:55.141637 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.141618 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:55.141731 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.141694 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:28:55.141811 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.141791 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 14:28:55.141945 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.141926 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-75tjn\"" Jul 06 14:28:55.142095 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.142077 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 14:28:55.142171 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.142100 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 14:28:55.142171 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.142159 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 14:28:55.142428 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.142183 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pv8dm\"" Jul 06 14:28:55.142428 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.142264 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 14:28:55.143854 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.143793 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.146330 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.146310 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:28:55.146433 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.146382 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:28:55.149012 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.148987 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.151838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.151458 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-96jvk\"" Jul 06 14:28:55.151838 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.151620 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:28:55.151838 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.151688 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:28:55.153052 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.153030 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 14:28:55.153249 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.153214 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 14:28:55.153336 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.153316 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 14:28:55.153591 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.153572 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 14:28:55.153706 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.153609 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 14:28:55.153706 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.153642 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 14:28:55.154404 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.154380 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 14:28:55.154480 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.154435 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 14:28:55.154544 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.154526 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gskn7\"" Jul 06 14:28:55.159698 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.159663 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 14:28:55.226980 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.226958 2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 14:28:55.228758 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.228735 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-48kzf" Jul 06 14:28:55.236369 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.236314 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-102.ec2.internal" event={"ID":"61c1ee201a8907e80dca74652a24fb31","Type":"ContainerStarted","Data":"ed542b0dda8209194a80344d6505385d0b0530660065a2fac2ae35bd8f15acb3"} Jul 06 14:28:55.237257 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237234 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7sv9\" (UniqueName: \"kubernetes.io/projected/45eec7f4-47b3-4a95-920c-ef8cc0a710c2-kube-api-access-h7sv9\") pod \"iptables-alerter-bz7tn\" (UID: \"45eec7f4-47b3-4a95-920c-ef8cc0a710c2\") " pod="openshift-network-operator/iptables-alerter-bz7tn" Jul 06 14:28:55.237382 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237282 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-cni-binary-copy\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.237382 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237317 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d071527c-3279-43ea-8b14-239560c6dc6e-cni-binary-copy\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.237382 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237345 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-run-netns\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.237382 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237369 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2j67\" (UniqueName: \"kubernetes.io/projected/d071527c-3279-43ea-8b14-239560c6dc6e-kube-api-access-z2j67\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.237653 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237419 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed76f900-7a58-48b0-8cf8-d53fea1737f0-agent-certs\") pod \"konnectivity-agent-75l8n\" (UID: \"ed76f900-7a58-48b0-8cf8-d53fea1737f0\") " pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:28:55.237653 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237448 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n57g8\" (UniqueName: \"kubernetes.io/projected/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-kube-api-access-n57g8\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.237653 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237473 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-kubernetes\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.237653 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237505 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-run-multus-certs\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.237653 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237532 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-etc-kubernetes\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.237653 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237554 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/888d055b-85a7-4b48-becd-0f12b488f994-tmp-dir\") pod \"node-resolver-vvm5c\" (UID: \"888d055b-85a7-4b48-becd-0f12b488f994\") " pod="openshift-dns/node-resolver-vvm5c" Jul 06 14:28:55.237653 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237576 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-sys-fs\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.237653 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237599 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-systemd\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.237653 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237624 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsgzv\" (UniqueName: \"kubernetes.io/projected/1f8b3a09-7749-4094-a2b8-fe09f23b1d44-kube-api-access-rsgzv\") pod \"node-ca-99szs\" (UID: \"1f8b3a09-7749-4094-a2b8-fe09f23b1d44\") " pod="openshift-image-registry/node-ca-99szs" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237658 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-node-log\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237683 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-cni-bin\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237714 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-os-release\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237737 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-kubelet\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237765 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b81dc945-3aca-47e3-aacc-476e6704f594-ovn-node-metrics-cert\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237790 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-multus-socket-dir-parent\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237818 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-run-k8s-cni-cncf-io\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237844 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-var-lib-kubelet\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237872 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-hostroot\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237898 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d071527c-3279-43ea-8b14-239560c6dc6e-multus-daemon-config\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237924 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-sys\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237951 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-slash\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.237978 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.238014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238005 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238038 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238063 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jchr6\" (UniqueName: \"kubernetes.io/projected/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-kube-api-access-jchr6\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238100 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-etc-openvswitch\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238129 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcxc6\" (UniqueName: \"kubernetes.io/projected/b81dc945-3aca-47e3-aacc-476e6704f594-kube-api-access-pcxc6\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238161 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-socket-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238187 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-sysconfig\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238208 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-sysctl-conf\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238232 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-lib-modules\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238254 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-tuned\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238301 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-run-systemd\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238343 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-cnibin\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238376 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-system-cni-dir\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238405 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-multus-conf-dir\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238431 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-run-netns\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238441 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:55.238736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238476 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238503 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b81dc945-3aca-47e3-aacc-476e6704f594-ovnkube-script-lib\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238528 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nwbf\" (UniqueName: \"kubernetes.io/projected/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-kube-api-access-9nwbf\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238554 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4lr2\" (UniqueName: \"kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2\") pod \"network-check-target-hc5qx\" (UID: \"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f\") " pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238579 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/888d055b-85a7-4b48-becd-0f12b488f994-hosts-file\") pod \"node-resolver-vvm5c\" (UID: \"888d055b-85a7-4b48-becd-0f12b488f994\") " pod="openshift-dns/node-resolver-vvm5c" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238778 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-system-cni-dir\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238377 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d071527c-3279-43ea-8b14-239560c6dc6e-cni-binary-copy\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238847 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-run-multus-certs\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238854 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-multus-conf-dir\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238550 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" event={"ID":"ecbb59684f17544ae54fb17e71a28592","Type":"ContainerStarted","Data":"824e91260244153e77d494e37351c3536a5fbf092d3c649ed0d092904f41ed46"} Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.238921 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-etc-kubernetes\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239065 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-multus-socket-dir-parent\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239124 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-run-k8s-cni-cncf-io\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239159 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-var-lib-kubelet\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239193 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-hostroot\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239373 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-registration-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239403 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-device-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.239570 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239427 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-sysctl-d\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239452 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239482 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239507 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-dbus\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239516 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d071527c-3279-43ea-8b14-239560c6dc6e-multus-daemon-config\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239530 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f8b3a09-7749-4094-a2b8-fe09f23b1d44-serviceca\") pod \"node-ca-99szs\" (UID: \"1f8b3a09-7749-4094-a2b8-fe09f23b1d44\") " pod="openshift-image-registry/node-ca-99szs" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239560 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-run-openvswitch\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239585 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-systemd-units\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239608 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b81dc945-3aca-47e3-aacc-476e6704f594-ovnkube-config\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239634 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-var-lib-cni-bin\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239657 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-cnibin\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239681 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-kubelet-config\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239703 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-run\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239727 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-tmp\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239753 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-var-lib-openvswitch\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239775 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-run-ovn-kubernetes\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.240448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239800 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-system-cni-dir\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239825 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh7zx\" (UniqueName: \"kubernetes.io/projected/888d055b-85a7-4b48-becd-0f12b488f994-kube-api-access-xh7zx\") pod \"node-resolver-vvm5c\" (UID: \"888d055b-85a7-4b48-becd-0f12b488f994\") " pod="openshift-dns/node-resolver-vvm5c" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239859 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239887 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-var-lib-cni-bin\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239888 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jctg\" (UniqueName: \"kubernetes.io/projected/feefb1dd-3073-46f1-bd64-a43792da5c4f-kube-api-access-4jctg\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239955 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed76f900-7a58-48b0-8cf8-d53fea1737f0-konnectivity-ca\") pod \"konnectivity-agent-75l8n\" (UID: \"ed76f900-7a58-48b0-8cf8-d53fea1737f0\") " pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.239973 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-cnibin\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240008 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-modprobe-d\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240033 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-run-netns\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240056 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f8b3a09-7749-4094-a2b8-fe09f23b1d44-host\") pod \"node-ca-99szs\" (UID: \"1f8b3a09-7749-4094-a2b8-fe09f23b1d44\") " pod="openshift-image-registry/node-ca-99szs" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240110 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-run-ovn\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240155 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-log-socket\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240187 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-var-lib-cni-multus\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240209 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-multus-cni-dir\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240235 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-os-release\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240274 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45eec7f4-47b3-4a95-920c-ef8cc0a710c2-iptables-alerter-script\") pod \"iptables-alerter-bz7tn\" (UID: \"45eec7f4-47b3-4a95-920c-ef8cc0a710c2\") " pod="openshift-network-operator/iptables-alerter-bz7tn" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240319 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-var-lib-kubelet\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.241200 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240342 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-host\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.242059 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240430 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-os-release\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.242059 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240473 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-host-var-lib-cni-multus\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.242059 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240509 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-cni-netd\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.242059 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240536 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b81dc945-3aca-47e3-aacc-476e6704f594-env-overrides\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.242059 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240564 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45eec7f4-47b3-4a95-920c-ef8cc0a710c2-host-slash\") pod \"iptables-alerter-bz7tn\" (UID: \"45eec7f4-47b3-4a95-920c-ef8cc0a710c2\") " pod="openshift-network-operator/iptables-alerter-bz7tn" Jul 06 14:28:55.242059 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240663 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d071527c-3279-43ea-8b14-239560c6dc6e-multus-cni-dir\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.242059 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240809 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/45eec7f4-47b3-4a95-920c-ef8cc0a710c2-host-slash\") pod \"iptables-alerter-bz7tn\" (UID: \"45eec7f4-47b3-4a95-920c-ef8cc0a710c2\") " pod="openshift-network-operator/iptables-alerter-bz7tn" Jul 06 14:28:55.242059 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.240861 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/45eec7f4-47b3-4a95-920c-ef8cc0a710c2-iptables-alerter-script\") pod \"iptables-alerter-bz7tn\" (UID: \"45eec7f4-47b3-4a95-920c-ef8cc0a710c2\") " pod="openshift-network-operator/iptables-alerter-bz7tn" Jul 06 14:28:55.244392 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.244370 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-48kzf" Jul 06 14:28:55.249355 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.249328 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 14:28:55.253203 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.253180 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2j67\" (UniqueName: \"kubernetes.io/projected/d071527c-3279-43ea-8b14-239560c6dc6e-kube-api-access-z2j67\") pod \"multus-vsjzf\" (UID: \"d071527c-3279-43ea-8b14-239560c6dc6e\") " pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.262354 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.262331 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7sv9\" (UniqueName: \"kubernetes.io/projected/45eec7f4-47b3-4a95-920c-ef8cc0a710c2-kube-api-access-h7sv9\") pod \"iptables-alerter-bz7tn\" (UID: \"45eec7f4-47b3-4a95-920c-ef8cc0a710c2\") " pod="openshift-network-operator/iptables-alerter-bz7tn" Jul 06 14:28:55.341556 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341229 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-cni-binary-copy\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.341556 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341285 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed76f900-7a58-48b0-8cf8-d53fea1737f0-agent-certs\") pod \"konnectivity-agent-75l8n\" (UID: \"ed76f900-7a58-48b0-8cf8-d53fea1737f0\") " pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:28:55.341556 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341313 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n57g8\" (UniqueName: \"kubernetes.io/projected/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-kube-api-access-n57g8\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.341556 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341340 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-kubernetes\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.341556 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341370 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/888d055b-85a7-4b48-becd-0f12b488f994-tmp-dir\") pod \"node-resolver-vvm5c\" (UID: \"888d055b-85a7-4b48-becd-0f12b488f994\") " pod="openshift-dns/node-resolver-vvm5c" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341605 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-sys-fs\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-systemd\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341682 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsgzv\" (UniqueName: \"kubernetes.io/projected/1f8b3a09-7749-4094-a2b8-fe09f23b1d44-kube-api-access-rsgzv\") pod \"node-ca-99szs\" (UID: \"1f8b3a09-7749-4094-a2b8-fe09f23b1d44\") " pod="openshift-image-registry/node-ca-99szs" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341707 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-node-log\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341719 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/888d055b-85a7-4b48-becd-0f12b488f994-tmp-dir\") pod \"node-resolver-vvm5c\" (UID: \"888d055b-85a7-4b48-becd-0f12b488f994\") " pod="openshift-dns/node-resolver-vvm5c" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341732 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-cni-bin\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341757 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-os-release\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341821 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-kubernetes\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341836 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-os-release\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341855 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-kubelet\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341884 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b81dc945-3aca-47e3-aacc-476e6704f594-ovn-node-metrics-cert\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.341919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341916 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-cni-binary-copy\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341942 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-systemd\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341922 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-sys\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341966 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-sys\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341887 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-node-log\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341950 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-sys-fs\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341989 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-slash\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342021 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-kubelet\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.341973 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-cni-bin\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342035 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342070 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-slash\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342115 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342166 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342274 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342272 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jchr6\" (UniqueName: \"kubernetes.io/projected/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-kube-api-access-jchr6\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342328 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-etc-openvswitch\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342358 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcxc6\" (UniqueName: \"kubernetes.io/projected/b81dc945-3aca-47e3-aacc-476e6704f594-kube-api-access-pcxc6\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.342550 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342386 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-socket-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342411 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-sysconfig\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342436 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-sysctl-conf\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342461 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-lib-modules\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342490 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-tuned\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342516 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-run-systemd\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342557 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-cnibin\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342586 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342616 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342626 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b81dc945-3aca-47e3-aacc-476e6704f594-ovnkube-script-lib\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342684 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nwbf\" (UniqueName: \"kubernetes.io/projected/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-kube-api-access-9nwbf\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342713 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-sysconfig\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342720 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-lib-modules\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342713 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342729 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lr2\" (UniqueName: \"kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2\") pod \"network-check-target-hc5qx\" (UID: \"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f\") " pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342761 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-run-systemd\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.343453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342770 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/888d055b-85a7-4b48-becd-0f12b488f994-hosts-file\") pod \"node-resolver-vvm5c\" (UID: \"888d055b-85a7-4b48-becd-0f12b488f994\") " pod="openshift-dns/node-resolver-vvm5c" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342828 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-registration-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.342851 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342842 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-socket-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342879 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-sysctl-conf\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342883 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-device-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342911 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-sysctl-d\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.342931 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret podName:2e25332d-2afd-48fd-98a6-c42e1a6f93b9 nodeName:}" failed. No retries permitted until 2026-07-06 14:28:55.8429099 +0000 UTC m=+3.483879812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret") pod "global-pull-secret-syncer-dzg6l" (UID: "2e25332d-2afd-48fd-98a6-c42e1a6f93b9") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342673 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-cnibin\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342586 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-etc-openvswitch\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342966 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342987 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-device-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.342998 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343071 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-sysctl-d\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343157 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343202 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-dbus\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343227 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.344311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343291 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f8b3a09-7749-4094-a2b8-fe09f23b1d44-serviceca\") pod \"node-ca-99szs\" (UID: \"1f8b3a09-7749-4094-a2b8-fe09f23b1d44\") " pod="openshift-image-registry/node-ca-99szs" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343322 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-run-openvswitch\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343349 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-systemd-units\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343375 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b81dc945-3aca-47e3-aacc-476e6704f594-ovnkube-config\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343427 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-kubelet-config\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343451 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-run\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343494 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-tmp\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343519 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-var-lib-openvswitch\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343538 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-dbus\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343549 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-run-ovn-kubernetes\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343597 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-kubelet-config\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343663 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-run-ovn-kubernetes\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343714 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-var-lib-openvswitch\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343716 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343757 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/888d055b-85a7-4b48-becd-0f12b488f994-hosts-file\") pod \"node-resolver-vvm5c\" (UID: \"888d055b-85a7-4b48-becd-0f12b488f994\") " pod="openshift-dns/node-resolver-vvm5c" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343783 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-run\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343791 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f8b3a09-7749-4094-a2b8-fe09f23b1d44-serviceca\") pod \"node-ca-99szs\" (UID: \"1f8b3a09-7749-4094-a2b8-fe09f23b1d44\") " pod="openshift-image-registry/node-ca-99szs" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343801 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b81dc945-3aca-47e3-aacc-476e6704f594-ovnkube-script-lib\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.345078 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343818 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-run-openvswitch\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343858 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-system-cni-dir\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343862 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-registration-dir\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343902 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7zx\" (UniqueName: \"kubernetes.io/projected/888d055b-85a7-4b48-becd-0f12b488f994-kube-api-access-xh7zx\") pod \"node-resolver-vvm5c\" (UID: \"888d055b-85a7-4b48-becd-0f12b488f994\") " pod="openshift-dns/node-resolver-vvm5c" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343914 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-system-cni-dir\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343957 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344080 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jctg\" (UniqueName: \"kubernetes.io/projected/feefb1dd-3073-46f1-bd64-a43792da5c4f-kube-api-access-4jctg\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344134 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed76f900-7a58-48b0-8cf8-d53fea1737f0-konnectivity-ca\") pod \"konnectivity-agent-75l8n\" (UID: \"ed76f900-7a58-48b0-8cf8-d53fea1737f0\") " pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.344250 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.343797 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-systemd-units\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.344397 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs podName:feefb1dd-3073-46f1-bd64-a43792da5c4f nodeName:}" failed. No retries permitted until 2026-07-06 14:28:55.844376645 +0000 UTC m=+3.485346559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs") pod "network-metrics-daemon-wdpvc" (UID: "feefb1dd-3073-46f1-bd64-a43792da5c4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344436 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-modprobe-d\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344482 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-run-netns\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344529 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-run-netns\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344559 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f8b3a09-7749-4094-a2b8-fe09f23b1d44-host\") pod \"node-ca-99szs\" (UID: \"1f8b3a09-7749-4094-a2b8-fe09f23b1d44\") " pod="openshift-image-registry/node-ca-99szs" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344591 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-modprobe-d\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344593 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-run-ovn\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.346718 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344620 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f8b3a09-7749-4094-a2b8-fe09f23b1d44-host\") pod \"node-ca-99szs\" (UID: \"1f8b3a09-7749-4094-a2b8-fe09f23b1d44\") " pod="openshift-image-registry/node-ca-99szs" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344631 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-log-socket\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344649 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b81dc945-3aca-47e3-aacc-476e6704f594-ovnkube-config\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344662 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-var-lib-kubelet\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344669 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-run-ovn\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344701 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-log-socket\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344704 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-host\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344738 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-cni-netd\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344740 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-var-lib-kubelet\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344743 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-host\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.344785 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b81dc945-3aca-47e3-aacc-476e6704f594-host-cni-netd\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.345219 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed76f900-7a58-48b0-8cf8-d53fea1737f0-konnectivity-ca\") pod \"konnectivity-agent-75l8n\" (UID: \"ed76f900-7a58-48b0-8cf8-d53fea1737f0\") " pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.345796 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b81dc945-3aca-47e3-aacc-476e6704f594-env-overrides\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.345881 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b81dc945-3aca-47e3-aacc-476e6704f594-env-overrides\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.345987 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-etc-tuned\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.346023 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b81dc945-3aca-47e3-aacc-476e6704f594-ovn-node-metrics-cert\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.347647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.346531 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed76f900-7a58-48b0-8cf8-d53fea1737f0-agent-certs\") pod \"konnectivity-agent-75l8n\" (UID: \"ed76f900-7a58-48b0-8cf8-d53fea1737f0\") " pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:28:55.348529 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.348505 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-tmp\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.354627 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.354606 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:28:55.354750 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.354634 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:28:55.354750 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.354649 2569 projected.go:196] Error preparing data for projected volume kube-api-access-m4lr2 for pod openshift-network-diagnostics/network-check-target-hc5qx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:28:55.354750 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.354716 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2 podName:05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f nodeName:}" failed. No retries permitted until 2026-07-06 14:28:55.854699378 +0000 UTC m=+3.495669292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m4lr2" (UniqueName: "kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2") pod "network-check-target-hc5qx" (UID: "05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:28:55.355617 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.355271 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcxc6\" (UniqueName: \"kubernetes.io/projected/b81dc945-3aca-47e3-aacc-476e6704f594-kube-api-access-pcxc6\") pod \"ovnkube-node-q7ks8\" (UID: \"b81dc945-3aca-47e3-aacc-476e6704f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.356573 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.356549 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n57g8\" (UniqueName: \"kubernetes.io/projected/987d1eb4-0a8a-4364-9d4e-3493eb694aaa-kube-api-access-n57g8\") pod \"aws-ebs-csi-driver-node-ml2cl\" (UID: \"987d1eb4-0a8a-4364-9d4e-3493eb694aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.357032 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.356991 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jchr6\" (UniqueName: \"kubernetes.io/projected/63fad9b1-c3ee-42e4-a83d-bb75533da8b3-kube-api-access-jchr6\") pod \"tuned-bzqcw\" (UID: \"63fad9b1-c3ee-42e4-a83d-bb75533da8b3\") " pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.357032 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.357020 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsgzv\" (UniqueName: \"kubernetes.io/projected/1f8b3a09-7749-4094-a2b8-fe09f23b1d44-kube-api-access-rsgzv\") pod \"node-ca-99szs\" (UID: \"1f8b3a09-7749-4094-a2b8-fe09f23b1d44\") " pod="openshift-image-registry/node-ca-99szs" Jul 06 14:28:55.363474 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.363338 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jctg\" (UniqueName: \"kubernetes.io/projected/feefb1dd-3073-46f1-bd64-a43792da5c4f-kube-api-access-4jctg\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:28:55.364510 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.364484 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh7zx\" (UniqueName: \"kubernetes.io/projected/888d055b-85a7-4b48-becd-0f12b488f994-kube-api-access-xh7zx\") pod \"node-resolver-vvm5c\" (UID: \"888d055b-85a7-4b48-becd-0f12b488f994\") " pod="openshift-dns/node-resolver-vvm5c" Jul 06 14:28:55.364603 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.364536 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nwbf\" (UniqueName: \"kubernetes.io/projected/4bbc83fb-4444-4495-ab75-bd3e8f1013ac-kube-api-access-9nwbf\") pod \"multus-additional-cni-plugins-bmxf6\" (UID: \"4bbc83fb-4444-4495-ab75-bd3e8f1013ac\") " pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.437374 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.437344 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vsjzf" Jul 06 14:28:55.448918 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.448890 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bz7tn" Jul 06 14:28:55.458553 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.458522 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" Jul 06 14:28:55.459310 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:55.459282 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45eec7f4_47b3_4a95_920c_ef8cc0a710c2.slice/crio-5e75ab492b4cf5566ea9672955cac93fa38c135682ba97ddd040ab307549df7d WatchSource:0}: Error finding container 5e75ab492b4cf5566ea9672955cac93fa38c135682ba97ddd040ab307549df7d: Status 404 returned error can't find the container with id 5e75ab492b4cf5566ea9672955cac93fa38c135682ba97ddd040ab307549df7d Jul 06 14:28:55.465563 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.465540 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" Jul 06 14:28:55.470035 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:55.470005 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod987d1eb4_0a8a_4364_9d4e_3493eb694aaa.slice/crio-dbe5b64df1b11ce10f14ee2cfc4608cbe8092be7b6918c3148f07bda4c01d3e7 WatchSource:0}: Error finding container dbe5b64df1b11ce10f14ee2cfc4608cbe8092be7b6918c3148f07bda4c01d3e7: Status 404 returned error can't find the container with id dbe5b64df1b11ce10f14ee2cfc4608cbe8092be7b6918c3148f07bda4c01d3e7 Jul 06 14:28:55.473737 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.473398 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vvm5c" Jul 06 14:28:55.479120 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:55.479086 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63fad9b1_c3ee_42e4_a83d_bb75533da8b3.slice/crio-368b170b5034fc23a67c283c31fd9650016388ed61f0731eeec700314b844245 WatchSource:0}: Error finding container 368b170b5034fc23a67c283c31fd9650016388ed61f0731eeec700314b844245: Status 404 returned error can't find the container with id 368b170b5034fc23a67c283c31fd9650016388ed61f0731eeec700314b844245 Jul 06 14:28:55.484497 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.484474 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-99szs" Jul 06 14:28:55.485551 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:55.485518 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod888d055b_85a7_4b48_becd_0f12b488f994.slice/crio-0412cfd1e3b1c128bdb8acb5c61df144aa5cbd6921ea8b24bf706dfb43e1b373 WatchSource:0}: Error finding container 0412cfd1e3b1c128bdb8acb5c61df144aa5cbd6921ea8b24bf706dfb43e1b373: Status 404 returned error can't find the container with id 0412cfd1e3b1c128bdb8acb5c61df144aa5cbd6921ea8b24bf706dfb43e1b373 Jul 06 14:28:55.495552 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:55.495524 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f8b3a09_7749_4094_a2b8_fe09f23b1d44.slice/crio-406347f659e4b35470e006c034c602b52a21ac169be87a4a57d4c120faf8fce9 WatchSource:0}: Error finding container 406347f659e4b35470e006c034c602b52a21ac169be87a4a57d4c120faf8fce9: Status 404 returned error can't find the container with id 406347f659e4b35470e006c034c602b52a21ac169be87a4a57d4c120faf8fce9 Jul 06 14:28:55.496261 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.496232 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:28:55.505894 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.505871 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:28:55.507905 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:55.507867 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded76f900_7a58_48b0_8cf8_d53fea1737f0.slice/crio-40b61ac1b3f1b5e1360c078aeca650c33ef59be4f7136842fcfdb87704670378 WatchSource:0}: Error finding container 40b61ac1b3f1b5e1360c078aeca650c33ef59be4f7136842fcfdb87704670378: Status 404 returned error can't find the container with id 40b61ac1b3f1b5e1360c078aeca650c33ef59be4f7136842fcfdb87704670378 Jul 06 14:28:55.513111 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.513062 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bmxf6" Jul 06 14:28:55.518971 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:55.518933 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb81dc945_3aca_47e3_aacc_476e6704f594.slice/crio-ca3d40c1610c62bb8ef973e39cccbcb0aea5d3fe7ef616b0a32fb4a6a1aca0f9 WatchSource:0}: Error finding container ca3d40c1610c62bb8ef973e39cccbcb0aea5d3fe7ef616b0a32fb4a6a1aca0f9: Status 404 returned error can't find the container with id ca3d40c1610c62bb8ef973e39cccbcb0aea5d3fe7ef616b0a32fb4a6a1aca0f9 Jul 06 14:28:55.528718 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:28:55.528691 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bbc83fb_4444_4495_ab75_bd3e8f1013ac.slice/crio-b722a23efbf35bd313029ae6dd605178043ddafd211a79f20d4224e5f89564f8 WatchSource:0}: Error finding container b722a23efbf35bd313029ae6dd605178043ddafd211a79f20d4224e5f89564f8: Status 404 returned error can't find the container with id b722a23efbf35bd313029ae6dd605178043ddafd211a79f20d4224e5f89564f8 Jul 06 14:28:55.683124 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.683044 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:28:55.849534 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.849490 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:55.849715 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.849566 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:28:55.849715 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.849656 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:28:55.849715 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.849692 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:28:55.849909 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.849739 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret podName:2e25332d-2afd-48fd-98a6-c42e1a6f93b9 nodeName:}" failed. No retries permitted until 2026-07-06 14:28:56.849721558 +0000 UTC m=+4.490691487 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret") pod "global-pull-secret-syncer-dzg6l" (UID: "2e25332d-2afd-48fd-98a6-c42e1a6f93b9") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:28:55.849909 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.849758 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs podName:feefb1dd-3073-46f1-bd64-a43792da5c4f nodeName:}" failed. No retries permitted until 2026-07-06 14:28:56.849749522 +0000 UTC m=+4.490719432 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs") pod "network-metrics-daemon-wdpvc" (UID: "feefb1dd-3073-46f1-bd64-a43792da5c4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:28:55.950509 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:55.950409 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lr2\" (UniqueName: \"kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2\") pod \"network-check-target-hc5qx\" (UID: \"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f\") " pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:28:55.950671 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.950572 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:28:55.950671 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.950596 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:28:55.950671 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.950610 2569 projected.go:196] Error preparing data for projected volume kube-api-access-m4lr2 for pod openshift-network-diagnostics/network-check-target-hc5qx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:28:55.950671 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:55.950669 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2 podName:05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f nodeName:}" failed. No retries permitted until 2026-07-06 14:28:56.950653463 +0000 UTC m=+4.591623389 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4lr2" (UniqueName: "kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2") pod "network-check-target-hc5qx" (UID: "05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:28:56.242253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.242170 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bz7tn" event={"ID":"45eec7f4-47b3-4a95-920c-ef8cc0a710c2","Type":"ContainerStarted","Data":"5e75ab492b4cf5566ea9672955cac93fa38c135682ba97ddd040ab307549df7d"} Jul 06 14:28:56.243674 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.243634 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vsjzf" event={"ID":"d071527c-3279-43ea-8b14-239560c6dc6e","Type":"ContainerStarted","Data":"f32d064b8a43628441f679d35a8063b1315c52ffd2972907d5ad50a8d7884da5"} Jul 06 14:28:56.244984 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.244960 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-75l8n" event={"ID":"ed76f900-7a58-48b0-8cf8-d53fea1737f0","Type":"ContainerStarted","Data":"40b61ac1b3f1b5e1360c078aeca650c33ef59be4f7136842fcfdb87704670378"} Jul 06 14:28:56.246256 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.246209 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmxf6" event={"ID":"4bbc83fb-4444-4495-ab75-bd3e8f1013ac","Type":"ContainerStarted","Data":"b722a23efbf35bd313029ae6dd605178043ddafd211a79f20d4224e5f89564f8"} Jul 06 14:28:56.247506 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.247463 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:23:55 +0000 UTC" deadline="2028-04-10 07:15:11.483348864 +0000 UTC" Jul 06 14:28:56.247626 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.247507 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15448h46m15.235846962s" Jul 06 14:28:56.247626 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.247553 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" event={"ID":"b81dc945-3aca-47e3-aacc-476e6704f594","Type":"ContainerStarted","Data":"ca3d40c1610c62bb8ef973e39cccbcb0aea5d3fe7ef616b0a32fb4a6a1aca0f9"} Jul 06 14:28:56.248892 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.248856 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-99szs" event={"ID":"1f8b3a09-7749-4094-a2b8-fe09f23b1d44","Type":"ContainerStarted","Data":"406347f659e4b35470e006c034c602b52a21ac169be87a4a57d4c120faf8fce9"} Jul 06 14:28:56.250185 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.250156 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vvm5c" event={"ID":"888d055b-85a7-4b48-becd-0f12b488f994","Type":"ContainerStarted","Data":"0412cfd1e3b1c128bdb8acb5c61df144aa5cbd6921ea8b24bf706dfb43e1b373"} Jul 06 14:28:56.251503 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.251482 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" event={"ID":"63fad9b1-c3ee-42e4-a83d-bb75533da8b3","Type":"ContainerStarted","Data":"368b170b5034fc23a67c283c31fd9650016388ed61f0731eeec700314b844245"} Jul 06 14:28:56.252792 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.252761 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" event={"ID":"987d1eb4-0a8a-4364-9d4e-3493eb694aaa","Type":"ContainerStarted","Data":"dbe5b64df1b11ce10f14ee2cfc4608cbe8092be7b6918c3148f07bda4c01d3e7"} Jul 06 14:28:56.857273 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.857139 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:28:56.857273 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.857242 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:56.857454 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:56.857314 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:28:56.857454 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:56.857371 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs podName:feefb1dd-3073-46f1-bd64-a43792da5c4f nodeName:}" failed. No retries permitted until 2026-07-06 14:28:58.857354975 +0000 UTC m=+6.498324891 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs") pod "network-metrics-daemon-wdpvc" (UID: "feefb1dd-3073-46f1-bd64-a43792da5c4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:28:56.857454 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:56.857431 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:28:56.857623 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:56.857494 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret podName:2e25332d-2afd-48fd-98a6-c42e1a6f93b9 nodeName:}" failed. No retries permitted until 2026-07-06 14:28:58.857477364 +0000 UTC m=+6.498447297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret") pod "global-pull-secret-syncer-dzg6l" (UID: "2e25332d-2afd-48fd-98a6-c42e1a6f93b9") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:28:56.957740 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:56.957699 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lr2\" (UniqueName: \"kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2\") pod \"network-check-target-hc5qx\" (UID: \"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f\") " pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:28:56.957923 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:56.957885 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:28:56.957923 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:56.957906 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:28:56.957923 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:56.957919 2569 projected.go:196] Error preparing data for projected volume kube-api-access-m4lr2 for pod openshift-network-diagnostics/network-check-target-hc5qx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:28:56.958082 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:56.957982 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2 podName:05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f nodeName:}" failed. No retries permitted until 2026-07-06 14:28:58.957961287 +0000 UTC m=+6.598931216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4lr2" (UniqueName: "kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2") pod "network-check-target-hc5qx" (UID: "05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:28:57.234183 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:57.233419 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:28:57.234183 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:57.233594 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:28:57.234183 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:57.233849 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:57.234183 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:57.233950 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:28:57.234183 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:57.234027 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:28:57.234183 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:57.234131 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:28:57.248620 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:57.248392 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:23:55 +0000 UTC" deadline="2028-01-04 22:52:24.023562113 +0000 UTC" Jul 06 14:28:57.248620 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:57.248435 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13136h23m26.775132596s" Jul 06 14:28:57.273708 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:57.273673 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-102.ec2.internal" event={"ID":"61c1ee201a8907e80dca74652a24fb31","Type":"ContainerStarted","Data":"ef8f2ee684e78b7295d87512295a69456ae69cf9a0e3b283b5c1e1c69896a180"} Jul 06 14:28:57.299873 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:57.299813 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-102.ec2.internal" podStartSLOduration=2.299792053 podStartE2EDuration="2.299792053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:28:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:57.299096792 +0000 UTC m=+4.940066723" watchObservedRunningTime="2026-07-06 14:28:57.299792053 +0000 UTC m=+4.940761988" Jul 06 14:28:58.302485 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:58.302440 2569 generic.go:356] "Generic (PLEG): container finished" podID="ecbb59684f17544ae54fb17e71a28592" containerID="3b3efedaf241d176e9e866def0afc669636a28a3dd4f2a351fecb800701e92a5" exitCode=0 Jul 06 14:28:58.303150 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:58.303122 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" event={"ID":"ecbb59684f17544ae54fb17e71a28592","Type":"ContainerDied","Data":"3b3efedaf241d176e9e866def0afc669636a28a3dd4f2a351fecb800701e92a5"} Jul 06 14:28:58.871589 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:58.871518 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:58.871808 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:58.871598 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:28:58.871808 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:58.871742 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:28:58.871808 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:58.871807 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs podName:feefb1dd-3073-46f1-bd64-a43792da5c4f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:02.871787296 +0000 UTC m=+10.512757210 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs") pod "network-metrics-daemon-wdpvc" (UID: "feefb1dd-3073-46f1-bd64-a43792da5c4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:28:58.872322 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:58.872163 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:28:58.872322 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:58.872239 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret podName:2e25332d-2afd-48fd-98a6-c42e1a6f93b9 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:02.872220355 +0000 UTC m=+10.513190280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret") pod "global-pull-secret-syncer-dzg6l" (UID: "2e25332d-2afd-48fd-98a6-c42e1a6f93b9") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:28:58.972027 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:58.971986 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lr2\" (UniqueName: \"kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2\") pod \"network-check-target-hc5qx\" (UID: \"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f\") " pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:28:58.972252 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:58.972202 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:28:58.972252 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:58.972224 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:28:58.972252 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:58.972238 2569 projected.go:196] Error preparing data for projected volume kube-api-access-m4lr2 for pod openshift-network-diagnostics/network-check-target-hc5qx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:28:58.972458 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:58.972323 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2 podName:05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:02.972303194 +0000 UTC m=+10.613273123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4lr2" (UniqueName: "kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2") pod "network-check-target-hc5qx" (UID: "05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:28:59.234952 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:59.234674 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:28:59.234952 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:59.234827 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:28:59.235540 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:59.235349 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:28:59.235540 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:59.235481 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:28:59.235716 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:28:59.235557 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:28:59.235768 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:28:59.235688 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:01.232438 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:01.232406 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:01.232911 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:01.232545 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:01.232992 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:01.232918 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:01.233061 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:01.233034 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:01.233124 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:01.233080 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:01.233175 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:01.233160 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:02.909157 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:02.909112 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:02.909657 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:02.909187 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:02.909657 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:02.909299 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:29:02.909657 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:02.909319 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:29:02.909657 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:02.909381 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs podName:feefb1dd-3073-46f1-bd64-a43792da5c4f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:10.909358325 +0000 UTC m=+18.550328262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs") pod "network-metrics-daemon-wdpvc" (UID: "feefb1dd-3073-46f1-bd64-a43792da5c4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:29:02.909657 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:02.909404 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret podName:2e25332d-2afd-48fd-98a6-c42e1a6f93b9 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:10.909393472 +0000 UTC m=+18.550363397 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret") pod "global-pull-secret-syncer-dzg6l" (UID: "2e25332d-2afd-48fd-98a6-c42e1a6f93b9") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:29:03.010332 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:03.010293 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lr2\" (UniqueName: \"kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2\") pod \"network-check-target-hc5qx\" (UID: \"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f\") " pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:03.010534 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:03.010512 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:29:03.010592 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:03.010536 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:29:03.010592 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:03.010550 2569 projected.go:196] Error preparing data for projected volume kube-api-access-m4lr2 for pod openshift-network-diagnostics/network-check-target-hc5qx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:29:03.010667 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:03.010604 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2 podName:05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:11.010587661 +0000 UTC m=+18.651557586 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4lr2" (UniqueName: "kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2") pod "network-check-target-hc5qx" (UID: "05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:29:03.232732 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:03.232641 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:03.232885 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:03.232760 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:03.233277 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:03.233248 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:03.233380 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:03.233249 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:03.233438 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:03.233390 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:03.233492 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:03.233470 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:05.232154 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:05.231662 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:05.232154 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:05.231691 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:05.232154 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:05.231792 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:05.232154 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:05.231886 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:05.232154 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:05.232042 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:05.232154 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:05.232137 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:07.236474 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:07.236254 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:07.236842 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:07.236650 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:07.236842 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:07.236804 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:07.237092 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:07.237065 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:07.237198 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:07.237143 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:07.237503 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:07.237406 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:08.331274 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.331236 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-99szs" event={"ID":"1f8b3a09-7749-4094-a2b8-fe09f23b1d44","Type":"ContainerStarted","Data":"f92d6d95de54dbcf2adc110497fe03639b9f0cc0d8362bf83aee60d51f1bf4d6"} Jul 06 14:29:08.332911 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.332882 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vvm5c" event={"ID":"888d055b-85a7-4b48-becd-0f12b488f994","Type":"ContainerStarted","Data":"b4fe7b42b01278b455d20cc1473038c0f522f11dc399c8269384fd23b88ff566"} Jul 06 14:29:08.334661 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.334635 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" event={"ID":"63fad9b1-c3ee-42e4-a83d-bb75533da8b3","Type":"ContainerStarted","Data":"10003a6ef7534b4ace8e2ee039ed5eaccb41bba1f4a28f60793e81c32a4c672b"} Jul 06 14:29:08.336313 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.336283 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" event={"ID":"987d1eb4-0a8a-4364-9d4e-3493eb694aaa","Type":"ContainerStarted","Data":"47a640773484683f16cc8425da9066aea678df86446fd63869a9bf2bc1dbff77"} Jul 06 14:29:08.338417 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.338392 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" event={"ID":"ecbb59684f17544ae54fb17e71a28592","Type":"ContainerStarted","Data":"3122a34cfd95101d5369875252220fc7162068f37665c77c416c887f4aaf9b0c"} Jul 06 14:29:08.340123 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.340098 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-75l8n" event={"ID":"ed76f900-7a58-48b0-8cf8-d53fea1737f0","Type":"ContainerStarted","Data":"a5f3e27289d86bc8c627de9760d094ba7318ac92d7a76f51c6cd1348c48ab1d2"} Jul 06 14:29:08.342002 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.341963 2569 generic.go:356] "Generic (PLEG): container finished" podID="4bbc83fb-4444-4495-ab75-bd3e8f1013ac" containerID="0038b6f3526a09ff5358b35649ebc2c2a24b106d398cec26af6e80df27620565" exitCode=0 Jul 06 14:29:08.342002 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.341991 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmxf6" event={"ID":"4bbc83fb-4444-4495-ab75-bd3e8f1013ac","Type":"ContainerDied","Data":"0038b6f3526a09ff5358b35649ebc2c2a24b106d398cec26af6e80df27620565"} Jul 06 14:29:08.357860 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.357812 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-99szs" podStartSLOduration=2.6501342340000003 podStartE2EDuration="14.357796943s" totalImagesPullingTime="11.707662709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:08.357125769 +0000 UTC m=+15.998095704" watchObservedRunningTime="2026-07-06 14:29:08.357796943 +0000 UTC m=+15.998766879" Jul 06 14:29:08.375080 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.375006 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-102.ec2.internal" podStartSLOduration=13.374990804 podStartE2EDuration="13.374990804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:28:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:08.374569333 +0000 UTC m=+16.015539287" watchObservedRunningTime="2026-07-06 14:29:08.374990804 +0000 UTC m=+16.015960739" Jul 06 14:29:08.399409 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.399368 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bzqcw" podStartSLOduration=2.674642138 podStartE2EDuration="14.399354519s" totalImagesPullingTime="11.724712381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:08.399317372 +0000 UTC m=+16.040287320" watchObservedRunningTime="2026-07-06 14:29:08.399354519 +0000 UTC m=+16.040324450" Jul 06 14:29:08.450472 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.450423 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vvm5c" podStartSLOduration=2.72868896 podStartE2EDuration="14.450404547s" totalImagesPullingTime="11.721715587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:08.450329671 +0000 UTC m=+16.091299607" watchObservedRunningTime="2026-07-06 14:29:08.450404547 +0000 UTC m=+16.091374478" Jul 06 14:29:08.472613 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:08.472564 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-75l8n" podStartSLOduration=2.778033613 podStartE2EDuration="14.472551587s" totalImagesPullingTime="11.694517974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:08.471954812 +0000 UTC m=+16.112924756" watchObservedRunningTime="2026-07-06 14:29:08.472551587 +0000 UTC m=+16.113521518" Jul 06 14:29:09.231317 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:09.231046 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:09.231484 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:09.231054 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:09.231484 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:09.231345 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:09.231484 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:09.231410 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:09.231484 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:09.231105 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:09.231694 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:09.231544 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:09.345609 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:09.345571 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bz7tn" event={"ID":"45eec7f4-47b3-4a95-920c-ef8cc0a710c2","Type":"ContainerStarted","Data":"0610165ade5fa5eb61f03d0cebe9d0a2ddf4c38a4c02f328d7ba2705aaaa92ed"} Jul 06 14:29:09.364441 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:09.364400 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bz7tn" podStartSLOduration=3.619176438 podStartE2EDuration="15.364385466s" totalImagesPullingTime="11.745209028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:09.36422909 +0000 UTC m=+17.005199020" watchObservedRunningTime="2026-07-06 14:29:09.364385466 +0000 UTC m=+17.005355402" Jul 06 14:29:10.970738 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:10.970705 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:10.971253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:10.970767 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:10.971253 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:10.970866 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:29:10.971253 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:10.970925 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret podName:2e25332d-2afd-48fd-98a6-c42e1a6f93b9 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:26.970911646 +0000 UTC m=+34.611881555 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret") pod "global-pull-secret-syncer-dzg6l" (UID: "2e25332d-2afd-48fd-98a6-c42e1a6f93b9") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:29:10.971253 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:10.970868 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:29:10.971253 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:10.971017 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs podName:feefb1dd-3073-46f1-bd64-a43792da5c4f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:26.970995951 +0000 UTC m=+34.611965864 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs") pod "network-metrics-daemon-wdpvc" (UID: "feefb1dd-3073-46f1-bd64-a43792da5c4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:29:11.071702 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:11.071663 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lr2\" (UniqueName: \"kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2\") pod \"network-check-target-hc5qx\" (UID: \"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f\") " pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:11.071872 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:11.071774 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:29:11.071872 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:11.071797 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:29:11.071872 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:11.071810 2569 projected.go:196] Error preparing data for projected volume kube-api-access-m4lr2 for pod openshift-network-diagnostics/network-check-target-hc5qx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:29:11.071872 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:11.071865 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2 podName:05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:27.071847673 +0000 UTC m=+34.712817582 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4lr2" (UniqueName: "kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2") pod "network-check-target-hc5qx" (UID: "05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:29:11.231090 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:11.230977 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:11.231090 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:11.230979 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:11.231321 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:11.230977 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:11.231321 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:11.231135 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:11.231321 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:11.231260 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:11.231475 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:11.231343 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:13.133737 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:13.133702 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:29:13.134273 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:13.134253 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:29:13.231993 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:13.231957 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:13.232166 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:13.232030 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:13.232166 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:13.232046 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:13.232166 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:13.232152 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:13.232359 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:13.232252 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:13.232359 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:13.232340 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:15.234492 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:15.234460 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:15.234990 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:15.234504 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:15.234990 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:15.234504 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:15.234990 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:15.234574 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:15.234990 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:15.234690 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:15.234990 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:15.234783 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:16.774400 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:16.774374 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 14:29:17.205133 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.205049 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T14:29:16.774394756Z","UUID":"9c67c6c2-196d-4231-9698-5aeb1a88e451","Handler":null,"Name":"","Endpoint":""} Jul 06 14:29:17.206744 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.206722 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 14:29:17.206869 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.206762 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 14:29:17.234902 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.234870 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:17.235047 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.234988 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:17.235047 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:17.234997 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:17.235047 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.235019 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:17.235266 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:17.235113 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:17.235266 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:17.235190 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:17.362991 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.362962 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:29:17.363411 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.363384 2569 generic.go:356] "Generic (PLEG): container finished" podID="b81dc945-3aca-47e3-aacc-476e6704f594" containerID="3329f2f9e3f77bc443f63c3bf2c4623f2fea6990044b10fde477656e5f2fbba2" exitCode=1 Jul 06 14:29:17.363529 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.363458 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" event={"ID":"b81dc945-3aca-47e3-aacc-476e6704f594","Type":"ContainerStarted","Data":"009a7d740e6577b1fa5566b6da9f5c9158907032334faa6032a0589f5c62869e"} Jul 06 14:29:17.363592 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.363547 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" event={"ID":"b81dc945-3aca-47e3-aacc-476e6704f594","Type":"ContainerStarted","Data":"dbd37739b229e6871d33844873879e87dd5d6b59c3d40742b9e64d2c517c3083"} Jul 06 14:29:17.363647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.363602 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" event={"ID":"b81dc945-3aca-47e3-aacc-476e6704f594","Type":"ContainerStarted","Data":"a66a1e088eb6e818df859a8850bccb72045fd1745a57133b40533546f1a0ac63"} Jul 06 14:29:17.363706 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.363652 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" event={"ID":"b81dc945-3aca-47e3-aacc-476e6704f594","Type":"ContainerStarted","Data":"11b6b0d72661c39fd802b342efebf90e227e660bead3d4d9d4aa3b3a53f1d507"} Jul 06 14:29:17.363759 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.363707 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" event={"ID":"b81dc945-3aca-47e3-aacc-476e6704f594","Type":"ContainerDied","Data":"3329f2f9e3f77bc443f63c3bf2c4623f2fea6990044b10fde477656e5f2fbba2"} Jul 06 14:29:17.363813 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.363759 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" event={"ID":"b81dc945-3aca-47e3-aacc-476e6704f594","Type":"ContainerStarted","Data":"f3bcf1d84cf9638b3968992431fe3b1014a77a836d4e41a420f7dc88c351a220"} Jul 06 14:29:17.365658 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.365634 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" event={"ID":"987d1eb4-0a8a-4364-9d4e-3493eb694aaa","Type":"ContainerStarted","Data":"0fac7cfb61c0b7456b6c544b85cb2a6809784e8008b32fd93aef64139686fd58"} Jul 06 14:29:17.367364 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.367340 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vsjzf" event={"ID":"d071527c-3279-43ea-8b14-239560c6dc6e","Type":"ContainerStarted","Data":"e5f97d7d5722fde7afcd551ad0c9d6ac870a916a8358cde920ebe3d82093eb61"} Jul 06 14:29:17.395179 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:17.395092 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vsjzf" podStartSLOduration=2.214053353 podStartE2EDuration="23.395078018s" totalImagesPullingTime="21.181024665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:17.394644439 +0000 UTC m=+25.035614370" watchObservedRunningTime="2026-07-06 14:29:17.395078018 +0000 UTC m=+25.036047949" Jul 06 14:29:19.234822 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:19.234736 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:19.234822 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:19.234748 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:19.234822 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:19.234676 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:19.235578 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:19.234851 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:19.235578 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:19.234993 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:19.235578 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:19.235098 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:19.373482 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:19.373249 2569 generic.go:356] "Generic (PLEG): container finished" podID="4bbc83fb-4444-4495-ab75-bd3e8f1013ac" containerID="8a186521999e94220a6ecfd680245b3e46bc1dbec4728336bf834704ae9c66d8" exitCode=0 Jul 06 14:29:19.373482 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:19.373283 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmxf6" event={"ID":"4bbc83fb-4444-4495-ab75-bd3e8f1013ac","Type":"ContainerDied","Data":"8a186521999e94220a6ecfd680245b3e46bc1dbec4728336bf834704ae9c66d8"} Jul 06 14:29:20.378602 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:20.378578 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:29:20.379054 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:20.378986 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" event={"ID":"b81dc945-3aca-47e3-aacc-476e6704f594","Type":"ContainerStarted","Data":"98662c55668a2e16d993142f081635a745f8dd411b9de805eb70d98f253ff13c"} Jul 06 14:29:20.380775 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:20.380749 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" event={"ID":"987d1eb4-0a8a-4364-9d4e-3493eb694aaa","Type":"ContainerStarted","Data":"0be23078d30bc268fe169b7877030f59e00be2ec7d35795074d56669f7e2900b"} Jul 06 14:29:20.407266 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:20.407220 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ml2cl" podStartSLOduration=2.16465643 podStartE2EDuration="26.407188773s" totalImagesPullingTime="24.242532343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:20.406976031 +0000 UTC m=+28.047945961" watchObservedRunningTime="2026-07-06 14:29:20.407188773 +0000 UTC m=+28.048158760" Jul 06 14:29:21.233419 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:21.233390 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:21.233419 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:21.233416 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:21.233568 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:21.233390 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:21.233568 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:21.233491 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:21.233635 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:21.233575 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:21.233668 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:21.233639 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:21.384316 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:21.384241 2569 generic.go:356] "Generic (PLEG): container finished" podID="4bbc83fb-4444-4495-ab75-bd3e8f1013ac" containerID="f29fd59e9c09a33d5c7c6cc6ad500ea9ebd63a6ba7459c024cb1d611a27e40ce" exitCode=0 Jul 06 14:29:21.384684 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:21.384325 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmxf6" event={"ID":"4bbc83fb-4444-4495-ab75-bd3e8f1013ac","Type":"ContainerDied","Data":"f29fd59e9c09a33d5c7c6cc6ad500ea9ebd63a6ba7459c024cb1d611a27e40ce"} Jul 06 14:29:22.389756 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:22.389724 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:29:22.390162 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:22.390090 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" event={"ID":"b81dc945-3aca-47e3-aacc-476e6704f594","Type":"ContainerStarted","Data":"c50ba22e309b430fd699777cae700eebd2a38acbc1e1012180a1cd220a614442"} Jul 06 14:29:22.390325 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:22.390304 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:29:22.390487 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:22.390376 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:29:22.390487 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:22.390428 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:29:22.390566 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:22.390540 2569 scope.go:117] "RemoveContainer" containerID="3329f2f9e3f77bc443f63c3bf2c4623f2fea6990044b10fde477656e5f2fbba2" Jul 06 14:29:22.405525 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:22.405366 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:29:22.405624 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:22.405605 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:29:23.231643 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:23.231610 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:23.231823 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:23.231694 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:23.231823 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:23.231706 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:23.231823 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:23.231784 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:23.231995 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:23.231830 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:23.231995 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:23.231883 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:23.365867 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:23.365839 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:29:23.366035 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:23.365942 2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 14:29:23.366445 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:23.366419 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-75l8n" Jul 06 14:29:23.395103 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:23.395076 2569 generic.go:356] "Generic (PLEG): container finished" podID="4bbc83fb-4444-4495-ab75-bd3e8f1013ac" containerID="4dbf3445aced586e554fb00ab671cc5e2214038d7a7b1416b840d6074f553fee" exitCode=0 Jul 06 14:29:23.395641 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:23.395157 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmxf6" event={"ID":"4bbc83fb-4444-4495-ab75-bd3e8f1013ac","Type":"ContainerDied","Data":"4dbf3445aced586e554fb00ab671cc5e2214038d7a7b1416b840d6074f553fee"} Jul 06 14:29:23.399024 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:23.399006 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:29:23.399419 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:23.399399 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" event={"ID":"b81dc945-3aca-47e3-aacc-476e6704f594","Type":"ContainerStarted","Data":"7f5351f922bdce16dc695f6babe46a0ed784abfc1c7b0a50e3949eb1e260cfac"} Jul 06 14:29:23.486698 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:23.486596 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" podStartSLOduration=8.327145347 podStartE2EDuration="29.486573726s" totalImagesPullingTime="21.159428379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:23.480424378 +0000 UTC m=+31.121394322" watchObservedRunningTime="2026-07-06 14:29:23.486573726 +0000 UTC m=+31.127543658" Jul 06 14:29:24.033826 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:24.033793 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dzg6l"] Jul 06 14:29:24.034006 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:24.033893 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:24.034006 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:24.033981 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:24.038556 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:24.037617 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hc5qx"] Jul 06 14:29:24.038556 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:24.037771 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:24.038556 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:24.037901 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:24.057729 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:24.057698 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wdpvc"] Jul 06 14:29:24.057864 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:24.057804 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:24.057929 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:24.057907 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:25.231684 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:25.231645 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:25.232348 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:25.231773 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:26.231778 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:26.231599 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:26.232367 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:26.231666 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:26.232367 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:26.231898 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:26.232367 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:26.231953 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:26.987535 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:26.987496 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:26.987739 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:26.987568 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:26.987739 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:26.987647 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:29:26.987739 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:26.987673 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:29:26.987739 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:26.987721 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret podName:2e25332d-2afd-48fd-98a6-c42e1a6f93b9 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:58.987699001 +0000 UTC m=+66.628668913 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret") pod "global-pull-secret-syncer-dzg6l" (UID: "2e25332d-2afd-48fd-98a6-c42e1a6f93b9") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:29:26.987739 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:26.987738 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs podName:feefb1dd-3073-46f1-bd64-a43792da5c4f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:58.987729519 +0000 UTC m=+66.628699427 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs") pod "network-metrics-daemon-wdpvc" (UID: "feefb1dd-3073-46f1-bd64-a43792da5c4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:29:27.087936 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:27.087905 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lr2\" (UniqueName: \"kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2\") pod \"network-check-target-hc5qx\" (UID: \"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f\") " pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:27.088117 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:27.088086 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:29:27.088198 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:27.088128 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:29:27.088198 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:27.088144 2569 projected.go:196] Error preparing data for projected volume kube-api-access-m4lr2 for pod openshift-network-diagnostics/network-check-target-hc5qx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:29:27.088301 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:27.088226 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2 podName:05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:59.088192839 +0000 UTC m=+66.729162761 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4lr2" (UniqueName: "kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2") pod "network-check-target-hc5qx" (UID: "05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:29:27.231318 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:27.231284 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:27.231501 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:27.231408 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hc5qx" podUID="05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f" Jul 06 14:29:28.231401 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:28.231369 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:28.231905 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:28.231489 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wdpvc" podUID="feefb1dd-3073-46f1-bd64-a43792da5c4f" Jul 06 14:29:28.231905 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:28.231543 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:28.231905 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:28.231654 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dzg6l" podUID="2e25332d-2afd-48fd-98a6-c42e1a6f93b9" Jul 06 14:29:29.129956 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.129930 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-102.ec2.internal" event="NodeReady" Jul 06 14:29:29.130090 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.130079 2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 14:29:29.172126 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.172093 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9db9f9db4-nk4ht"] Jul 06 14:29:29.194809 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.194784 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9db9f9db4-nk4ht"] Jul 06 14:29:29.194915 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.194826 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-czjlx"] Jul 06 14:29:29.194975 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.194945 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.198465 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.198440 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 14:29:29.198558 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.198528 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 14:29:29.198628 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.198441 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 14:29:29.199482 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.199315 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vnmkm\"" Jul 06 14:29:29.204809 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.204790 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 14:29:29.213319 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.213300 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rnk6j"] Jul 06 14:29:29.213447 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.213431 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.215781 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.215761 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 14:29:29.215878 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.215832 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rs6zp\"" Jul 06 14:29:29.215974 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.215949 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 14:29:29.231290 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.231265 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-czjlx"] Jul 06 14:29:29.231416 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.231314 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rnk6j"] Jul 06 14:29:29.231416 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.231391 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rnk6j" Jul 06 14:29:29.231824 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.231595 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:29.234266 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.234247 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 14:29:29.234537 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.234511 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 14:29:29.234625 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.234545 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 14:29:29.234818 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.234802 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kwxdr\"" Jul 06 14:29:29.234897 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.234832 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 14:29:29.234897 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.234852 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 14:29:29.235095 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.235080 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tspsl\"" Jul 06 14:29:29.304461 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304434 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vprjw\" (UniqueName: \"kubernetes.io/projected/c27dfc49-d472-4046-a1ac-55191f07e7c1-kube-api-access-vprjw\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.304548 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304478 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-installation-pull-secrets\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.304548 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304539 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c27dfc49-d472-4046-a1ac-55191f07e7c1-config-volume\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.304613 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304567 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.304613 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304604 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-image-registry-private-configuration\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.304678 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304625 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-ca-trust-extracted\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.304710 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304679 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-trusted-ca\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.304739 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304710 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-bound-sa-token\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.304739 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304733 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fq2m\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-kube-api-access-4fq2m\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.304801 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304750 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.304801 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304784 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c27dfc49-d472-4046-a1ac-55191f07e7c1-tmp-dir\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.304860 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.304812 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-certificates\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.405650 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.405619 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-bound-sa-token\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.405791 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.405654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fq2m\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-kube-api-access-4fq2m\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.405791 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.405675 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.405958 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.405939 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:29:29.406000 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.405978 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c27dfc49-d472-4046-a1ac-55191f07e7c1-tmp-dir\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.406000 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.405999 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls podName:c27dfc49-d472-4046-a1ac-55191f07e7c1 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:29.905981916 +0000 UTC m=+37.546951828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls") pod "dns-default-czjlx" (UID: "c27dfc49-d472-4046-a1ac-55191f07e7c1") : secret "dns-default-metrics-tls" not found Jul 06 14:29:29.406090 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406027 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-certificates\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.406090 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406058 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0dae0635-5d0a-47eb-994a-14f09891a285-cert\") pod \"ingress-canary-rnk6j\" (UID: \"0dae0635-5d0a-47eb-994a-14f09891a285\") " pod="openshift-ingress-canary/ingress-canary-rnk6j" Jul 06 14:29:29.406090 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406074 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvx6h\" (UniqueName: \"kubernetes.io/projected/0dae0635-5d0a-47eb-994a-14f09891a285-kube-api-access-cvx6h\") pod \"ingress-canary-rnk6j\" (UID: \"0dae0635-5d0a-47eb-994a-14f09891a285\") " pod="openshift-ingress-canary/ingress-canary-rnk6j" Jul 06 14:29:29.406274 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406146 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprjw\" (UniqueName: \"kubernetes.io/projected/c27dfc49-d472-4046-a1ac-55191f07e7c1-kube-api-access-vprjw\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.406274 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406189 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-installation-pull-secrets\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.406393 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406278 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c27dfc49-d472-4046-a1ac-55191f07e7c1-config-volume\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.406393 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406297 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c27dfc49-d472-4046-a1ac-55191f07e7c1-tmp-dir\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.406393 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406317 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.406393 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406361 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-image-registry-private-configuration\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.406393 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406387 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-ca-trust-extracted\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.406665 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.406421 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:29:29.406665 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406430 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-trusted-ca\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.406665 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.406435 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db9f9db4-nk4ht: secret "image-registry-tls" not found Jul 06 14:29:29.406665 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.406559 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls podName:5f791f18-dd8e-46d2-8e38-cbdc9e26f75f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:29.906542813 +0000 UTC m=+37.547512732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls") pod "image-registry-9db9f9db4-nk4ht" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f") : secret "image-registry-tls" not found Jul 06 14:29:29.406665 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406627 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-certificates\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.406819 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406697 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-ca-trust-extracted\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.406819 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.406777 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c27dfc49-d472-4046-a1ac-55191f07e7c1-config-volume\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.407127 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.407097 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-trusted-ca\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.409924 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.409902 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-installation-pull-secrets\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.410011 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.409951 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-image-registry-private-configuration\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.414952 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.414924 2569 generic.go:356] "Generic (PLEG): container finished" podID="4bbc83fb-4444-4495-ab75-bd3e8f1013ac" containerID="e26584971d9b80f6da7cedeb2f0f82db6955353a557e4fd56c3d107c85a15c7a" exitCode=0 Jul 06 14:29:29.415058 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.414993 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmxf6" event={"ID":"4bbc83fb-4444-4495-ab75-bd3e8f1013ac","Type":"ContainerDied","Data":"e26584971d9b80f6da7cedeb2f0f82db6955353a557e4fd56c3d107c85a15c7a"} Jul 06 14:29:29.422558 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.422540 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vprjw\" (UniqueName: \"kubernetes.io/projected/c27dfc49-d472-4046-a1ac-55191f07e7c1-kube-api-access-vprjw\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.426757 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.426730 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fq2m\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-kube-api-access-4fq2m\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.430764 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.430747 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-bound-sa-token\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.507396 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.507360 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0dae0635-5d0a-47eb-994a-14f09891a285-cert\") pod \"ingress-canary-rnk6j\" (UID: \"0dae0635-5d0a-47eb-994a-14f09891a285\") " pod="openshift-ingress-canary/ingress-canary-rnk6j" Jul 06 14:29:29.507533 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.507407 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvx6h\" (UniqueName: \"kubernetes.io/projected/0dae0635-5d0a-47eb-994a-14f09891a285-kube-api-access-cvx6h\") pod \"ingress-canary-rnk6j\" (UID: \"0dae0635-5d0a-47eb-994a-14f09891a285\") " pod="openshift-ingress-canary/ingress-canary-rnk6j" Jul 06 14:29:29.510048 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.510026 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0dae0635-5d0a-47eb-994a-14f09891a285-cert\") pod \"ingress-canary-rnk6j\" (UID: \"0dae0635-5d0a-47eb-994a-14f09891a285\") " pod="openshift-ingress-canary/ingress-canary-rnk6j" Jul 06 14:29:29.531039 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.531016 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j"] Jul 06 14:29:29.550345 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.550290 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvx6h\" (UniqueName: \"kubernetes.io/projected/0dae0635-5d0a-47eb-994a-14f09891a285-kube-api-access-cvx6h\") pod \"ingress-canary-rnk6j\" (UID: \"0dae0635-5d0a-47eb-994a-14f09891a285\") " pod="openshift-ingress-canary/ingress-canary-rnk6j" Jul 06 14:29:29.551809 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.551788 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-lsf6f"] Jul 06 14:29:29.551919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.551905 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:29:29.565981 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.565957 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn"] Jul 06 14:29:29.566082 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.566026 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:29:29.566134 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.566094 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.566492 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.566471 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wdbkt\"" Jul 06 14:29:29.566587 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.566565 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 14:29:29.566677 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.566661 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 14:29:29.580330 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.580305 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-brjmq"] Jul 06 14:29:29.580427 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.580389 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:29.581711 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.581690 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 14:29:29.581832 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.581714 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:29:29.581986 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.581931 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 14:29:29.582069 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.581992 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 14:29:29.582265 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.582240 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hjr86\"" Jul 06 14:29:29.583548 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.583532 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 14:29:29.583756 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.583600 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jzn5z\"" Jul 06 14:29:29.585146 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.585129 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 14:29:29.585282 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.585168 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 14:29:29.585372 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.585358 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 14:29:29.592557 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.592539 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j"] Jul 06 14:29:29.592622 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.592570 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-lsf6f"] Jul 06 14:29:29.592622 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.592584 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-brjmq"] Jul 06 14:29:29.592622 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.592603 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn"] Jul 06 14:29:29.592720 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.592640 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-brjmq" Jul 06 14:29:29.608074 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.608050 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:29:29.608263 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.608247 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-xjddj\"" Jul 06 14:29:29.617928 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.617903 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 14:29:29.620677 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.620661 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 14:29:29.646512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.646493 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw"] Jul 06 14:29:29.650907 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.650889 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-m4bxb"] Jul 06 14:29:29.651012 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.650997 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" Jul 06 14:29:29.655076 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.655061 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m4bxb" Jul 06 14:29:29.658636 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.658613 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 14:29:29.658907 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.658891 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 14:29:29.659295 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.659279 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 14:29:29.659468 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.659453 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5fj92\"" Jul 06 14:29:29.659689 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.659676 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6fd947d8df-9l5bt"] Jul 06 14:29:29.660756 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.660740 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-d5266\"" Jul 06 14:29:29.665030 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.665013 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-5nw2p"] Jul 06 14:29:29.665128 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.665115 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.670800 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.670782 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 14:29:29.671013 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.670996 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 14:29:29.671268 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.671251 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.671381 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.671365 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 14:29:29.671745 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.671456 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n2g9b\"" Jul 06 14:29:29.671823 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.671506 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 14:29:29.671823 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.671578 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 14:29:29.675371 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.675357 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:29:29.676329 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.676304 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 14:29:29.676512 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.676497 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 14:29:29.676643 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.676524 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 14:29:29.676729 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.676713 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fmpmp\"" Jul 06 14:29:29.677562 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.677428 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 14:29:29.678098 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.678080 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-m4bxb"] Jul 06 14:29:29.682351 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.682333 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 14:29:29.682595 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.682569 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw"] Jul 06 14:29:29.690810 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.690792 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 14:29:29.693502 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.693479 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-5nw2p"] Jul 06 14:29:29.704189 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.704168 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6fd947d8df-9l5bt"] Jul 06 14:29:29.708668 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.708649 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmxch\" (UniqueName: \"kubernetes.io/projected/eafea937-7719-47d7-9240-e94089cdfcc4-kube-api-access-bmxch\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.708745 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.708676 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hwtb\" (UniqueName: \"kubernetes.io/projected/7e898dfd-d2ea-40c1-8d51-2141622b10f7-kube-api-access-5hwtb\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:29:29.708745 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.708693 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:29.708745 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.708711 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eafea937-7719-47d7-9240-e94089cdfcc4-config\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.708844 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.708763 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqlgn\" (UniqueName: \"kubernetes.io/projected/f9424d51-1719-4cfd-a85d-8163121ac834-kube-api-access-cqlgn\") pod \"volume-data-source-validator-5d456fdbcd-brjmq\" (UID: \"f9424d51-1719-4cfd-a85d-8163121ac834\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-brjmq" Jul 06 14:29:29.708883 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.708851 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eafea937-7719-47d7-9240-e94089cdfcc4-serving-cert\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.708883 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.708871 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eafea937-7719-47d7-9240-e94089cdfcc4-trusted-ca\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.708946 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.708893 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:29.708946 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.708922 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:29:29.708946 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.708937 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79rbl\" (UniqueName: \"kubernetes.io/projected/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-kube-api-access-79rbl\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:29.733758 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.733733 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb"] Jul 06 14:29:29.739423 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.739407 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" Jul 06 14:29:29.742118 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.742101 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 06 14:29:29.742733 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.742716 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 06 14:29:29.742825 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.742740 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 06 14:29:29.742825 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.742752 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:29:29.742825 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.742769 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-blgc2\"" Jul 06 14:29:29.748709 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.748689 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb"] Jul 06 14:29:29.809378 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809298 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86455cea-7f7b-47d6-a140-2809930847c8-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.809378 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809341 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b5kd\" (UniqueName: \"kubernetes.io/projected/86455cea-7f7b-47d6-a140-2809930847c8-kube-api-access-5b5kd\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.809565 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809402 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:29:29.809565 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809437 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/86455cea-7f7b-47d6-a140-2809930847c8-snapshots\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.809565 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809465 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-stats-auth\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.809565 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809489 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5fb1a5f8-6190-4002-9c4f-a36897910ac3-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-l2xzw\" (UID: \"5fb1a5f8-6190-4002-9c4f-a36897910ac3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" Jul 06 14:29:29.809565 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.809522 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:29:29.809721 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.809583 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls podName:7e898dfd-d2ea-40c1-8d51-2141622b10f7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:30.309569121 +0000 UTC m=+37.950539033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-j8r5j" (UID: "7e898dfd-d2ea-40c1-8d51-2141622b10f7") : secret "samples-operator-tls" not found Jul 06 14:29:29.809721 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809625 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eafea937-7719-47d7-9240-e94089cdfcc4-trusted-ca\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.809721 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809651 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86455cea-7f7b-47d6-a140-2809930847c8-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.809721 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809685 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbb7c\" (UniqueName: \"kubernetes.io/projected/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-kube-api-access-dbb7c\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.809844 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809724 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hwtb\" (UniqueName: \"kubernetes.io/projected/7e898dfd-d2ea-40c1-8d51-2141622b10f7-kube-api-access-5hwtb\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:29:29.809844 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809802 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqlgn\" (UniqueName: \"kubernetes.io/projected/f9424d51-1719-4cfd-a85d-8163121ac834-kube-api-access-cqlgn\") pod \"volume-data-source-validator-5d456fdbcd-brjmq\" (UID: \"f9424d51-1719-4cfd-a85d-8163121ac834\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-brjmq" Jul 06 14:29:29.809844 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809829 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-default-certificate\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.809936 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809847 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps8pk\" (UniqueName: \"kubernetes.io/projected/223e9073-108b-4ace-9200-96bb48086655-kube-api-access-ps8pk\") pod \"network-check-source-989f8b8dc-m4bxb\" (UID: \"223e9073-108b-4ace-9200-96bb48086655\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m4bxb" Jul 06 14:29:29.809936 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809876 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.810029 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809943 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eafea937-7719-47d7-9240-e94089cdfcc4-serving-cert\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.810029 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809966 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86455cea-7f7b-47d6-a140-2809930847c8-serving-cert\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.810029 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.809999 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl4zl\" (UniqueName: \"kubernetes.io/projected/5fb1a5f8-6190-4002-9c4f-a36897910ac3-kube-api-access-rl4zl\") pod \"service-ca-operator-5d46d9ffd4-l2xzw\" (UID: \"5fb1a5f8-6190-4002-9c4f-a36897910ac3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" Jul 06 14:29:29.810164 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.810028 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:29.810164 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.810050 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79rbl\" (UniqueName: \"kubernetes.io/projected/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-kube-api-access-79rbl\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:29.810164 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.810069 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmxch\" (UniqueName: \"kubernetes.io/projected/eafea937-7719-47d7-9240-e94089cdfcc4-kube-api-access-bmxch\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.810164 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.810088 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86455cea-7f7b-47d6-a140-2809930847c8-tmp\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.810164 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.810110 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:29.810164 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.810134 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5fb1a5f8-6190-4002-9c4f-a36897910ac3-config\") pod \"service-ca-operator-5d46d9ffd4-l2xzw\" (UID: \"5fb1a5f8-6190-4002-9c4f-a36897910ac3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" Jul 06 14:29:29.810481 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.810184 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eafea937-7719-47d7-9240-e94089cdfcc4-config\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.810481 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.810221 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.810481 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.810242 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:29.810481 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.810350 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls podName:fae66ff7-10e5-4713-a0a7-a2bfb0246aba nodeName:}" failed. No retries permitted until 2026-07-06 14:29:30.310325452 +0000 UTC m=+37.951295365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-vs5xn" (UID: "fae66ff7-10e5-4713-a0a7-a2bfb0246aba") : secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:29.810752 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.810731 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eafea937-7719-47d7-9240-e94089cdfcc4-trusted-ca\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.810816 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.810767 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:29.810816 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.810775 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eafea937-7719-47d7-9240-e94089cdfcc4-config\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.812393 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.812372 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eafea937-7719-47d7-9240-e94089cdfcc4-serving-cert\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.820956 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.820932 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hwtb\" (UniqueName: \"kubernetes.io/projected/7e898dfd-d2ea-40c1-8d51-2141622b10f7-kube-api-access-5hwtb\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:29:29.821789 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.821730 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79rbl\" (UniqueName: \"kubernetes.io/projected/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-kube-api-access-79rbl\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:29.821789 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.821734 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmxch\" (UniqueName: \"kubernetes.io/projected/eafea937-7719-47d7-9240-e94089cdfcc4-kube-api-access-bmxch\") pod \"console-operator-68b757fb67-lsf6f\" (UID: \"eafea937-7719-47d7-9240-e94089cdfcc4\") " pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.822297 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.822279 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqlgn\" (UniqueName: \"kubernetes.io/projected/f9424d51-1719-4cfd-a85d-8163121ac834-kube-api-access-cqlgn\") pod \"volume-data-source-validator-5d456fdbcd-brjmq\" (UID: \"f9424d51-1719-4cfd-a85d-8163121ac834\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-brjmq" Jul 06 14:29:29.841561 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.841541 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rnk6j" Jul 06 14:29:29.879584 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.879180 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:29.911548 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.911522 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-default-certificate\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.911680 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.911556 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps8pk\" (UniqueName: \"kubernetes.io/projected/223e9073-108b-4ace-9200-96bb48086655-kube-api-access-ps8pk\") pod \"network-check-source-989f8b8dc-m4bxb\" (UID: \"223e9073-108b-4ace-9200-96bb48086655\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m4bxb" Jul 06 14:29:29.911680 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.911576 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.911680 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.911602 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:29.911680 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.911619 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86455cea-7f7b-47d6-a140-2809930847c8-serving-cert\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.912014 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.911723 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:29:29.912014 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.911740 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db9f9db4-nk4ht: secret "image-registry-tls" not found Jul 06 14:29:29.912014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.911754 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17cfe1b2-63bb-4b47-b4c9-82f9ff59c333-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-mqcxb\" (UID: \"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" Jul 06 14:29:29.912014 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.911783 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:30.411761114 +0000 UTC m=+38.052731056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : configmap references non-existent config key: service-ca.crt Jul 06 14:29:29.912014 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.911826 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls podName:5f791f18-dd8e-46d2-8e38-cbdc9e26f75f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:30.911814012 +0000 UTC m=+38.552783936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls") pod "image-registry-9db9f9db4-nk4ht" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f") : secret "image-registry-tls" not found Jul 06 14:29:29.912014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.911860 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl4zl\" (UniqueName: \"kubernetes.io/projected/5fb1a5f8-6190-4002-9c4f-a36897910ac3-kube-api-access-rl4zl\") pod \"service-ca-operator-5d46d9ffd4-l2xzw\" (UID: \"5fb1a5f8-6190-4002-9c4f-a36897910ac3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" Jul 06 14:29:29.912014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.911911 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86455cea-7f7b-47d6-a140-2809930847c8-tmp\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.912014 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.911955 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5fb1a5f8-6190-4002-9c4f-a36897910ac3-config\") pod \"service-ca-operator-5d46d9ffd4-l2xzw\" (UID: \"5fb1a5f8-6190-4002-9c4f-a36897910ac3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912037 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/17cfe1b2-63bb-4b47-b4c9-82f9ff59c333-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-mqcxb\" (UID: \"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912071 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.912142 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912152 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6klwh\" (UniqueName: \"kubernetes.io/projected/17cfe1b2-63bb-4b47-b4c9-82f9ff59c333-kube-api-access-6klwh\") pod \"kube-storage-version-migrator-operator-846bf668f7-mqcxb\" (UID: \"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.912186 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:30.41217021 +0000 UTC m=+38.053140120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : secret "router-metrics-certs-default" not found Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912248 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86455cea-7f7b-47d6-a140-2809930847c8-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912280 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b5kd\" (UniqueName: \"kubernetes.io/projected/86455cea-7f7b-47d6-a140-2809930847c8-kube-api-access-5b5kd\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912292 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86455cea-7f7b-47d6-a140-2809930847c8-tmp\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912333 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/86455cea-7f7b-47d6-a140-2809930847c8-snapshots\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912355 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-stats-auth\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912378 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5fb1a5f8-6190-4002-9c4f-a36897910ac3-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-l2xzw\" (UID: \"5fb1a5f8-6190-4002-9c4f-a36897910ac3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912420 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86455cea-7f7b-47d6-a140-2809930847c8-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.912444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912449 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:29.912972 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912475 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbb7c\" (UniqueName: \"kubernetes.io/projected/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-kube-api-access-dbb7c\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.912972 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912643 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5fb1a5f8-6190-4002-9c4f-a36897910ac3-config\") pod \"service-ca-operator-5d46d9ffd4-l2xzw\" (UID: \"5fb1a5f8-6190-4002-9c4f-a36897910ac3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" Jul 06 14:29:29.912972 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.912752 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:29:29.912972 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:29.912801 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls podName:c27dfc49-d472-4046-a1ac-55191f07e7c1 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:30.912780093 +0000 UTC m=+38.553750026 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls") pod "dns-default-czjlx" (UID: "c27dfc49-d472-4046-a1ac-55191f07e7c1") : secret "dns-default-metrics-tls" not found Jul 06 14:29:29.913099 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.912991 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86455cea-7f7b-47d6-a140-2809930847c8-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.913099 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.913089 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/86455cea-7f7b-47d6-a140-2809930847c8-snapshots\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.913218 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.913171 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86455cea-7f7b-47d6-a140-2809930847c8-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.914607 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.914585 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86455cea-7f7b-47d6-a140-2809930847c8-serving-cert\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.915019 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.915001 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-stats-auth\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.915303 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.915282 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5fb1a5f8-6190-4002-9c4f-a36897910ac3-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-l2xzw\" (UID: \"5fb1a5f8-6190-4002-9c4f-a36897910ac3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" Jul 06 14:29:29.915414 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.915399 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-default-certificate\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.923526 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.923500 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl4zl\" (UniqueName: \"kubernetes.io/projected/5fb1a5f8-6190-4002-9c4f-a36897910ac3-kube-api-access-rl4zl\") pod \"service-ca-operator-5d46d9ffd4-l2xzw\" (UID: \"5fb1a5f8-6190-4002-9c4f-a36897910ac3\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" Jul 06 14:29:29.923905 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.923881 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps8pk\" (UniqueName: \"kubernetes.io/projected/223e9073-108b-4ace-9200-96bb48086655-kube-api-access-ps8pk\") pod \"network-check-source-989f8b8dc-m4bxb\" (UID: \"223e9073-108b-4ace-9200-96bb48086655\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m4bxb" Jul 06 14:29:29.924085 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.924070 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b5kd\" (UniqueName: \"kubernetes.io/projected/86455cea-7f7b-47d6-a140-2809930847c8-kube-api-access-5b5kd\") pod \"insights-operator-5c847dcfc4-5nw2p\" (UID: \"86455cea-7f7b-47d6-a140-2809930847c8\") " pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:29.925238 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.925214 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbb7c\" (UniqueName: \"kubernetes.io/projected/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-kube-api-access-dbb7c\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:29.950113 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.950087 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-brjmq" Jul 06 14:29:29.961457 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.961379 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" Jul 06 14:29:29.964824 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.964801 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m4bxb" Jul 06 14:29:29.980678 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:29.980647 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" Jul 06 14:29:30.014477 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.013762 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/17cfe1b2-63bb-4b47-b4c9-82f9ff59c333-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-mqcxb\" (UID: \"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" Jul 06 14:29:30.014477 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.013814 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6klwh\" (UniqueName: \"kubernetes.io/projected/17cfe1b2-63bb-4b47-b4c9-82f9ff59c333-kube-api-access-6klwh\") pod \"kube-storage-version-migrator-operator-846bf668f7-mqcxb\" (UID: \"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" Jul 06 14:29:30.014477 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.014171 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17cfe1b2-63bb-4b47-b4c9-82f9ff59c333-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-mqcxb\" (UID: \"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" Jul 06 14:29:30.014708 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.014682 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/17cfe1b2-63bb-4b47-b4c9-82f9ff59c333-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-mqcxb\" (UID: \"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" Jul 06 14:29:30.017037 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.017010 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/17cfe1b2-63bb-4b47-b4c9-82f9ff59c333-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-mqcxb\" (UID: \"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" Jul 06 14:29:30.025991 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.024937 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6klwh\" (UniqueName: \"kubernetes.io/projected/17cfe1b2-63bb-4b47-b4c9-82f9ff59c333-kube-api-access-6klwh\") pod \"kube-storage-version-migrator-operator-846bf668f7-mqcxb\" (UID: \"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" Jul 06 14:29:30.047767 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.047666 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" Jul 06 14:29:30.057959 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.057930 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-lsf6f"] Jul 06 14:29:30.062678 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.061989 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rnk6j"] Jul 06 14:29:30.184023 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.183765 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-brjmq"] Jul 06 14:29:30.187650 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:29:30.187618 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9424d51_1719_4cfd_a85d_8163121ac834.slice/crio-fc8c78a8628b874b15fb395d6b33281ff0adc48a03a47bf5915e13d891a3fbb2 WatchSource:0}: Error finding container fc8c78a8628b874b15fb395d6b33281ff0adc48a03a47bf5915e13d891a3fbb2: Status 404 returned error can't find the container with id fc8c78a8628b874b15fb395d6b33281ff0adc48a03a47bf5915e13d891a3fbb2 Jul 06 14:29:30.226475 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.226412 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-5nw2p"] Jul 06 14:29:30.228948 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:29:30.228922 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86455cea_7f7b_47d6_a140_2809930847c8.slice/crio-5884d370e3ceac455d3aa8bbb585282816acb6dff339dca1fd2e7c7a8b32463b WatchSource:0}: Error finding container 5884d370e3ceac455d3aa8bbb585282816acb6dff339dca1fd2e7c7a8b32463b: Status 404 returned error can't find the container with id 5884d370e3ceac455d3aa8bbb585282816acb6dff339dca1fd2e7c7a8b32463b Jul 06 14:29:30.231037 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.231023 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:30.231347 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.231175 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:30.233544 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.233526 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 14:29:30.234121 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.233947 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s774m\"" Jul 06 14:29:30.234121 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.234000 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 14:29:30.241974 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.241954 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-m4bxb"] Jul 06 14:29:30.244758 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:29:30.244726 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223e9073_108b_4ace_9200_96bb48086655.slice/crio-cc2bfc34d5dfa30fb4ded824052bd30a90a7f51c349b916c7a6096d97d8b912d WatchSource:0}: Error finding container cc2bfc34d5dfa30fb4ded824052bd30a90a7f51c349b916c7a6096d97d8b912d: Status 404 returned error can't find the container with id cc2bfc34d5dfa30fb4ded824052bd30a90a7f51c349b916c7a6096d97d8b912d Jul 06 14:29:30.245026 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.245003 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw"] Jul 06 14:29:30.247672 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:29:30.247648 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb1a5f8_6190_4002_9c4f_a36897910ac3.slice/crio-0f80d52e0ec721210996d41c9e0032b375b6aaa8465e61ced84a7350beb13d6c WatchSource:0}: Error finding container 0f80d52e0ec721210996d41c9e0032b375b6aaa8465e61ced84a7350beb13d6c: Status 404 returned error can't find the container with id 0f80d52e0ec721210996d41c9e0032b375b6aaa8465e61ced84a7350beb13d6c Jul 06 14:29:30.250321 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.250303 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb"] Jul 06 14:29:30.253463 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:29:30.253440 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17cfe1b2_63bb_4b47_b4c9_82f9ff59c333.slice/crio-8986a84eb2c4d286ff7cd2db779b7651e127164367df604b682a3d2ed55c53ae WatchSource:0}: Error finding container 8986a84eb2c4d286ff7cd2db779b7651e127164367df604b682a3d2ed55c53ae: Status 404 returned error can't find the container with id 8986a84eb2c4d286ff7cd2db779b7651e127164367df604b682a3d2ed55c53ae Jul 06 14:29:30.326727 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.326637 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:30.326727 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.326719 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:29:30.326936 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.326827 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:30.326936 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.326892 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls podName:fae66ff7-10e5-4713-a0a7-a2bfb0246aba nodeName:}" failed. No retries permitted until 2026-07-06 14:29:31.326873838 +0000 UTC m=+38.967843761 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-vs5xn" (UID: "fae66ff7-10e5-4713-a0a7-a2bfb0246aba") : secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:30.326936 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.326891 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:29:30.327070 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.326958 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls podName:7e898dfd-d2ea-40c1-8d51-2141622b10f7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:31.326939306 +0000 UTC m=+38.967909233 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-j8r5j" (UID: "7e898dfd-d2ea-40c1-8d51-2141622b10f7") : secret "samples-operator-tls" not found Jul 06 14:29:30.418513 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.418475 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" event={"ID":"86455cea-7f7b-47d6-a140-2809930847c8","Type":"ContainerStarted","Data":"5884d370e3ceac455d3aa8bbb585282816acb6dff339dca1fd2e7c7a8b32463b"} Jul 06 14:29:30.419554 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.419525 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m4bxb" event={"ID":"223e9073-108b-4ace-9200-96bb48086655","Type":"ContainerStarted","Data":"cc2bfc34d5dfa30fb4ded824052bd30a90a7f51c349b916c7a6096d97d8b912d"} Jul 06 14:29:30.420502 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.420475 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" event={"ID":"5fb1a5f8-6190-4002-9c4f-a36897910ac3","Type":"ContainerStarted","Data":"0f80d52e0ec721210996d41c9e0032b375b6aaa8465e61ced84a7350beb13d6c"} Jul 06 14:29:30.421493 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.421475 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-brjmq" event={"ID":"f9424d51-1719-4cfd-a85d-8163121ac834","Type":"ContainerStarted","Data":"fc8c78a8628b874b15fb395d6b33281ff0adc48a03a47bf5915e13d891a3fbb2"} Jul 06 14:29:30.422500 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.422479 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" event={"ID":"eafea937-7719-47d7-9240-e94089cdfcc4","Type":"ContainerStarted","Data":"9fbe7778675b0122c2dedff9ccd98c2fcf3506567ced37653c017c97e7fee647"} Jul 06 14:29:30.423490 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.423471 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" event={"ID":"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333","Type":"ContainerStarted","Data":"8986a84eb2c4d286ff7cd2db779b7651e127164367df604b682a3d2ed55c53ae"} Jul 06 14:29:30.426107 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.426086 2569 generic.go:356] "Generic (PLEG): container finished" podID="4bbc83fb-4444-4495-ab75-bd3e8f1013ac" containerID="7f31a3ff933f45722382b42ad35d1fc53ce2be20d46f5edbf6f397adcb03dba9" exitCode=0 Jul 06 14:29:30.426225 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.426162 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmxf6" event={"ID":"4bbc83fb-4444-4495-ab75-bd3e8f1013ac","Type":"ContainerDied","Data":"7f31a3ff933f45722382b42ad35d1fc53ce2be20d46f5edbf6f397adcb03dba9"} Jul 06 14:29:30.427173 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.427153 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:30.427280 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.427254 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:30.427280 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.427150 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rnk6j" event={"ID":"0dae0635-5d0a-47eb-994a-14f09891a285","Type":"ContainerStarted","Data":"8036c73894e504f311542760d9cbed4d61482174bd32814f1110c3612059de4e"} Jul 06 14:29:30.427398 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.427275 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:31.427256199 +0000 UTC m=+39.068226126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : configmap references non-existent config key: service-ca.crt Jul 06 14:29:30.427398 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.427338 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 14:29:30.427398 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.427384 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:31.427367932 +0000 UTC m=+39.068337850 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : secret "router-metrics-certs-default" not found Jul 06 14:29:30.932949 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.932105 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:30.932949 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:30.932220 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:30.932949 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.932387 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:29:30.932949 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.932402 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db9f9db4-nk4ht: secret "image-registry-tls" not found Jul 06 14:29:30.932949 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.932461 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls podName:5f791f18-dd8e-46d2-8e38-cbdc9e26f75f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:32.932442192 +0000 UTC m=+40.573412118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls") pod "image-registry-9db9f9db4-nk4ht" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f") : secret "image-registry-tls" not found Jul 06 14:29:30.932949 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.932866 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:29:30.932949 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:30.932912 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls podName:c27dfc49-d472-4046-a1ac-55191f07e7c1 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:32.932898169 +0000 UTC m=+40.573868082 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls") pod "dns-default-czjlx" (UID: "c27dfc49-d472-4046-a1ac-55191f07e7c1") : secret "dns-default-metrics-tls" not found Jul 06 14:29:31.336444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:31.335593 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:31.336444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:31.335687 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:29:31.336444 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:31.335864 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:29:31.336444 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:31.335928 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls podName:7e898dfd-d2ea-40c1-8d51-2141622b10f7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:33.335908755 +0000 UTC m=+40.976878667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-j8r5j" (UID: "7e898dfd-d2ea-40c1-8d51-2141622b10f7") : secret "samples-operator-tls" not found Jul 06 14:29:31.336444 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:31.336338 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:31.336444 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:31.336386 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls podName:fae66ff7-10e5-4713-a0a7-a2bfb0246aba nodeName:}" failed. No retries permitted until 2026-07-06 14:29:33.336372304 +0000 UTC m=+40.977342215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-vs5xn" (UID: "fae66ff7-10e5-4713-a0a7-a2bfb0246aba") : secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:31.436390 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:31.436354 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:31.436577 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:31.436458 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:31.436787 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:31.436627 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 14:29:31.436787 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:31.436691 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:33.436673426 +0000 UTC m=+41.077643338 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : secret "router-metrics-certs-default" not found Jul 06 14:29:31.436787 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:31.436753 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:33.436733556 +0000 UTC m=+41.077703468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : configmap references non-existent config key: service-ca.crt Jul 06 14:29:31.446494 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:31.446462 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmxf6" event={"ID":"4bbc83fb-4444-4495-ab75-bd3e8f1013ac","Type":"ContainerStarted","Data":"37d11cb6950852637de3a44dad4033b8fe806b678f212862eaeac7f2d97bbc00"} Jul 06 14:29:31.477831 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:31.477770 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bmxf6" podStartSLOduration=6.359587201 podStartE2EDuration="37.477748872s" totalImagesPullingTime="31.118161671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:31.475582914 +0000 UTC m=+39.116552846" watchObservedRunningTime="2026-07-06 14:29:31.477748872 +0000 UTC m=+39.118718828" Jul 06 14:29:32.453274 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:32.453185 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-brjmq" event={"ID":"f9424d51-1719-4cfd-a85d-8163121ac834","Type":"ContainerStarted","Data":"fe7c91406b76adaadd1a6e1e5be15758a802c47cd0080c9addd76895f397b0fc"} Jul 06 14:29:32.472664 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:32.472574 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-brjmq" podStartSLOduration=1.854228389 podStartE2EDuration="3.472553482s" totalImagesPullingTime="1.618325093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:32.47119923 +0000 UTC m=+40.112169161" watchObservedRunningTime="2026-07-06 14:29:32.472553482 +0000 UTC m=+40.113523417" Jul 06 14:29:32.952753 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:32.952722 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:32.952936 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:32.952795 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:32.952936 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:32.952891 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:29:32.953046 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:32.952954 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls podName:c27dfc49-d472-4046-a1ac-55191f07e7c1 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:36.952936823 +0000 UTC m=+44.593906741 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls") pod "dns-default-czjlx" (UID: "c27dfc49-d472-4046-a1ac-55191f07e7c1") : secret "dns-default-metrics-tls" not found Jul 06 14:29:32.953046 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:32.952970 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:29:32.953046 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:32.952993 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db9f9db4-nk4ht: secret "image-registry-tls" not found Jul 06 14:29:32.953192 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:32.953050 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls podName:5f791f18-dd8e-46d2-8e38-cbdc9e26f75f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:36.95303579 +0000 UTC m=+44.594005718 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls") pod "image-registry-9db9f9db4-nk4ht" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f") : secret "image-registry-tls" not found Jul 06 14:29:33.357380 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:33.357339 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:33.357565 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:33.357422 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:29:33.357565 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:33.357519 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:33.357565 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:33.357551 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:29:33.357703 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:33.357599 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls podName:fae66ff7-10e5-4713-a0a7-a2bfb0246aba nodeName:}" failed. No retries permitted until 2026-07-06 14:29:37.357582878 +0000 UTC m=+44.998552790 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-vs5xn" (UID: "fae66ff7-10e5-4713-a0a7-a2bfb0246aba") : secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:33.357703 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:33.357635 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls podName:7e898dfd-d2ea-40c1-8d51-2141622b10f7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:37.357625491 +0000 UTC m=+44.998595400 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-j8r5j" (UID: "7e898dfd-d2ea-40c1-8d51-2141622b10f7") : secret "samples-operator-tls" not found Jul 06 14:29:33.458741 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:33.458701 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:33.459187 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:33.458800 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:33.459187 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:33.458820 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:37.458800437 +0000 UTC m=+45.099770363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : configmap references non-existent config key: service-ca.crt Jul 06 14:29:33.459187 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:33.458872 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 14:29:33.459187 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:33.458915 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:37.458904255 +0000 UTC m=+45.099874165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : secret "router-metrics-certs-default" not found Jul 06 14:29:36.990535 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:36.990484 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:36.990979 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:36.990579 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:36.990979 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:36.990658 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:29:36.990979 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:36.990698 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:29:36.990979 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:36.990711 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db9f9db4-nk4ht: secret "image-registry-tls" not found Jul 06 14:29:36.990979 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:36.990730 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls podName:c27dfc49-d472-4046-a1ac-55191f07e7c1 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:44.990712136 +0000 UTC m=+52.631682066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls") pod "dns-default-czjlx" (UID: "c27dfc49-d472-4046-a1ac-55191f07e7c1") : secret "dns-default-metrics-tls" not found Jul 06 14:29:36.990979 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:36.990750 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls podName:5f791f18-dd8e-46d2-8e38-cbdc9e26f75f nodeName:}" failed. No retries permitted until 2026-07-06 14:29:44.990738796 +0000 UTC m=+52.631708718 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls") pod "image-registry-9db9f9db4-nk4ht" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f") : secret "image-registry-tls" not found Jul 06 14:29:37.394631 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:37.394452 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:29:37.394775 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:37.394610 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:29:37.394775 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:37.394722 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:37.394775 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:37.394749 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls podName:7e898dfd-d2ea-40c1-8d51-2141622b10f7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:45.394734419 +0000 UTC m=+53.035704331 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-j8r5j" (UID: "7e898dfd-d2ea-40c1-8d51-2141622b10f7") : secret "samples-operator-tls" not found Jul 06 14:29:37.394878 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:37.394801 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:37.394878 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:37.394844 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls podName:fae66ff7-10e5-4713-a0a7-a2bfb0246aba nodeName:}" failed. No retries permitted until 2026-07-06 14:29:45.394833227 +0000 UTC m=+53.035803136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-vs5xn" (UID: "fae66ff7-10e5-4713-a0a7-a2bfb0246aba") : secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:37.495753 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:37.495724 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:37.495898 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:37.495884 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:37.495898 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:37.495888 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 14:29:37.496019 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:37.495954 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:45.495935196 +0000 UTC m=+53.136905113 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : secret "router-metrics-certs-default" not found Jul 06 14:29:37.496019 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:37.495974 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:45.495964024 +0000 UTC m=+53.136933934 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : configmap references non-existent config key: service-ca.crt Jul 06 14:29:38.472530 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.472087 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" event={"ID":"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333","Type":"ContainerStarted","Data":"81e5113786915d5479bcf5b2b87e9e2cc7098491b3a5109ff6616e5d5e5e88e4"} Jul 06 14:29:38.473825 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.473775 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rnk6j" event={"ID":"0dae0635-5d0a-47eb-994a-14f09891a285","Type":"ContainerStarted","Data":"f99aafc70b1cab14d417f7213afb1e407666e202f87ba45888093f81a05034bc"} Jul 06 14:29:38.475529 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.475483 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" event={"ID":"86455cea-7f7b-47d6-a140-2809930847c8","Type":"ContainerStarted","Data":"9ed81b8877800a36dc64bf967d660e59b7731d47a3d44582d089bd2fb2023ce7"} Jul 06 14:29:38.477910 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.477857 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m4bxb" event={"ID":"223e9073-108b-4ace-9200-96bb48086655","Type":"ContainerStarted","Data":"ef31569ac952ab1a2cc116403f7ab17078243ac6dc6f109964a31419a5525638"} Jul 06 14:29:38.479786 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.479745 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" event={"ID":"5fb1a5f8-6190-4002-9c4f-a36897910ac3","Type":"ContainerStarted","Data":"86c04038b5446cd7db42e2abe4574b40aebcadc97d3061477e1afbeb41324bb0"} Jul 06 14:29:38.481898 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.481871 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/0.log" Jul 06 14:29:38.482010 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.481918 2569 generic.go:356] "Generic (PLEG): container finished" podID="eafea937-7719-47d7-9240-e94089cdfcc4" containerID="ca26b9671f299edb84c8d837d17d41bd77ea16f829d53b848b56092acd7edb45" exitCode=255 Jul 06 14:29:38.482010 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.481947 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" event={"ID":"eafea937-7719-47d7-9240-e94089cdfcc4","Type":"ContainerDied","Data":"ca26b9671f299edb84c8d837d17d41bd77ea16f829d53b848b56092acd7edb45"} Jul 06 14:29:38.482229 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.482199 2569 scope.go:117] "RemoveContainer" containerID="ca26b9671f299edb84c8d837d17d41bd77ea16f829d53b848b56092acd7edb45" Jul 06 14:29:38.495688 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.495649 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" podStartSLOduration=2.118028084 podStartE2EDuration="9.495632691s" totalImagesPullingTime="7.377604607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:38.494826424 +0000 UTC m=+46.135796370" watchObservedRunningTime="2026-07-06 14:29:38.495632691 +0000 UTC m=+46.136602619" Jul 06 14:29:38.517793 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.517626 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rnk6j" podStartSLOduration=1.9537996560000002 podStartE2EDuration="9.517607965s" totalImagesPullingTime="7.563808309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:38.516705829 +0000 UTC m=+46.157675750" watchObservedRunningTime="2026-07-06 14:29:38.517607965 +0000 UTC m=+46.158577899" Jul 06 14:29:38.559050 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.558990 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" podStartSLOduration=2.174424996 podStartE2EDuration="9.558969521s" totalImagesPullingTime="7.384544525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:38.557872582 +0000 UTC m=+46.198842514" watchObservedRunningTime="2026-07-06 14:29:38.558969521 +0000 UTC m=+46.199939453" Jul 06 14:29:38.579720 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.579600 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" podStartSLOduration=2.176229728 podStartE2EDuration="9.57958006s" totalImagesPullingTime="7.403350332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:38.578556003 +0000 UTC m=+46.219525936" watchObservedRunningTime="2026-07-06 14:29:38.57958006 +0000 UTC m=+46.220549993" Jul 06 14:29:38.599619 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:38.599565 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-m4bxb" podStartSLOduration=2.202728276 podStartE2EDuration="9.599549477s" totalImagesPullingTime="7.396821201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:38.598505557 +0000 UTC m=+46.239475486" watchObservedRunningTime="2026-07-06 14:29:38.599549477 +0000 UTC m=+46.240519408" Jul 06 14:29:39.486982 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.486957 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/1.log" Jul 06 14:29:39.487454 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.487405 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/0.log" Jul 06 14:29:39.487454 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.487435 2569 generic.go:356] "Generic (PLEG): container finished" podID="eafea937-7719-47d7-9240-e94089cdfcc4" containerID="3225dfb3bfd7235f377b76bb16b653319bc809b7420854111604f1bf0cbfc1b6" exitCode=255 Jul 06 14:29:39.487540 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.487514 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" event={"ID":"eafea937-7719-47d7-9240-e94089cdfcc4","Type":"ContainerDied","Data":"3225dfb3bfd7235f377b76bb16b653319bc809b7420854111604f1bf0cbfc1b6"} Jul 06 14:29:39.487588 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.487563 2569 scope.go:117] "RemoveContainer" containerID="ca26b9671f299edb84c8d837d17d41bd77ea16f829d53b848b56092acd7edb45" Jul 06 14:29:39.487813 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.487794 2569 scope.go:117] "RemoveContainer" containerID="3225dfb3bfd7235f377b76bb16b653319bc809b7420854111604f1bf0cbfc1b6" Jul 06 14:29:39.488010 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:39.487988 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-lsf6f_openshift-console-operator(eafea937-7719-47d7-9240-e94089cdfcc4)\"" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" podUID="eafea937-7719-47d7-9240-e94089cdfcc4" Jul 06 14:29:39.773440 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.773130 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k5bxg"] Jul 06 14:29:39.787790 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.787763 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.790728 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.790691 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 14:29:39.791000 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.790980 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 14:29:39.791000 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.790994 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rzjwm\"" Jul 06 14:29:39.792297 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.792277 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k5bxg"] Jul 06 14:29:39.821457 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.821430 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.821540 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.821490 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea26ce84-0bf5-4578-a800-983f5646b2b7-data-volume\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.821540 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.821525 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.821624 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.821544 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzd5d\" (UniqueName: \"kubernetes.io/projected/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-api-access-vzd5d\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.821624 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.821575 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ea26ce84-0bf5-4578-a800-983f5646b2b7-crio-socket\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.879982 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.879956 2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:39.880127 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.880019 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:39.922294 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.922264 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.922420 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.922324 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea26ce84-0bf5-4578-a800-983f5646b2b7-data-volume\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.922420 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.922369 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.922420 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.922388 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzd5d\" (UniqueName: \"kubernetes.io/projected/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-api-access-vzd5d\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.922420 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:39.922390 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:29:39.922593 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.922436 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ea26ce84-0bf5-4578-a800-983f5646b2b7-crio-socket\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.922593 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:39.922460 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm podName:ea26ce84-0bf5-4578-a800-983f5646b2b7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:40.422436962 +0000 UTC m=+48.063406872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k5bxg" (UID: "ea26ce84-0bf5-4578-a800-983f5646b2b7") : configmap "kube-rbac-proxy" not found Jul 06 14:29:39.922593 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:39.922523 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:29:39.922593 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:39.922584 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls podName:ea26ce84-0bf5-4578-a800-983f5646b2b7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:40.422567875 +0000 UTC m=+48.063537804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5bxg" (UID: "ea26ce84-0bf5-4578-a800-983f5646b2b7") : secret "insights-runtime-extractor-tls" not found Jul 06 14:29:39.922746 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.922660 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ea26ce84-0bf5-4578-a800-983f5646b2b7-crio-socket\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.922746 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.922717 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea26ce84-0bf5-4578-a800-983f5646b2b7-data-volume\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:39.933898 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:39.933870 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzd5d\" (UniqueName: \"kubernetes.io/projected/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-api-access-vzd5d\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:40.426572 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:40.426518 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:40.426787 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:40.426638 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:40.426787 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:40.426644 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:29:40.426787 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:40.426703 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:29:40.426787 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:40.426751 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm podName:ea26ce84-0bf5-4578-a800-983f5646b2b7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:41.42673338 +0000 UTC m=+49.067703305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k5bxg" (UID: "ea26ce84-0bf5-4578-a800-983f5646b2b7") : configmap "kube-rbac-proxy" not found Jul 06 14:29:40.426787 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:40.426770 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls podName:ea26ce84-0bf5-4578-a800-983f5646b2b7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:41.426761821 +0000 UTC m=+49.067731730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5bxg" (UID: "ea26ce84-0bf5-4578-a800-983f5646b2b7") : secret "insights-runtime-extractor-tls" not found Jul 06 14:29:40.491988 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:40.491960 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/1.log" Jul 06 14:29:40.492443 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:40.492404 2569 scope.go:117] "RemoveContainer" containerID="3225dfb3bfd7235f377b76bb16b653319bc809b7420854111604f1bf0cbfc1b6" Jul 06 14:29:40.492584 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:40.492562 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-lsf6f_openshift-console-operator(eafea937-7719-47d7-9240-e94089cdfcc4)\"" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" podUID="eafea937-7719-47d7-9240-e94089cdfcc4" Jul 06 14:29:41.435168 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:41.435129 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:41.435333 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:41.435292 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:29:41.435371 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:41.435327 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:41.435371 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:41.435353 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm podName:ea26ce84-0bf5-4578-a800-983f5646b2b7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:43.43533636 +0000 UTC m=+51.076306272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k5bxg" (UID: "ea26ce84-0bf5-4578-a800-983f5646b2b7") : configmap "kube-rbac-proxy" not found Jul 06 14:29:41.435465 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:41.435406 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:29:41.435465 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:41.435461 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls podName:ea26ce84-0bf5-4578-a800-983f5646b2b7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:43.435449034 +0000 UTC m=+51.076418943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5bxg" (UID: "ea26ce84-0bf5-4578-a800-983f5646b2b7") : secret "insights-runtime-extractor-tls" not found Jul 06 14:29:41.495489 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:41.495467 2569 scope.go:117] "RemoveContainer" containerID="3225dfb3bfd7235f377b76bb16b653319bc809b7420854111604f1bf0cbfc1b6" Jul 06 14:29:41.495825 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:41.495613 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-lsf6f_openshift-console-operator(eafea937-7719-47d7-9240-e94089cdfcc4)\"" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" podUID="eafea937-7719-47d7-9240-e94089cdfcc4" Jul 06 14:29:42.787918 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:42.787892 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vvm5c_888d055b-85a7-4b48-becd-0f12b488f994/dns-node-resolver/0.log" Jul 06 14:29:43.172904 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:43.172816 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-99szs_1f8b3a09-7749-4094-a2b8-fe09f23b1d44/node-ca/0.log" Jul 06 14:29:43.454583 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:43.454493 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:43.454743 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:43.454625 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:29:43.454743 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:43.454669 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:43.454743 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:43.454687 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm podName:ea26ce84-0bf5-4578-a800-983f5646b2b7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:47.454671311 +0000 UTC m=+55.095641220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k5bxg" (UID: "ea26ce84-0bf5-4578-a800-983f5646b2b7") : configmap "kube-rbac-proxy" not found Jul 06 14:29:43.454907 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:43.454788 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:29:43.454907 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:43.454844 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls podName:ea26ce84-0bf5-4578-a800-983f5646b2b7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:47.454827847 +0000 UTC m=+55.095797782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5bxg" (UID: "ea26ce84-0bf5-4578-a800-983f5646b2b7") : secret "insights-runtime-extractor-tls" not found Jul 06 14:29:44.372279 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:44.372228 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rnk6j_0dae0635-5d0a-47eb-994a-14f09891a285/serve-healthcheck-canary/0.log" Jul 06 14:29:44.976783 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:44.976745 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-mqcxb_17cfe1b2-63bb-4b47-b4c9-82f9ff59c333/kube-storage-version-migrator-operator/0.log" Jul 06 14:29:45.070318 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:45.070282 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:29:45.070494 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:45.070405 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:29:45.070494 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.070435 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:29:45.070494 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.070463 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db9f9db4-nk4ht: secret "image-registry-tls" not found Jul 06 14:29:45.070604 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.070518 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:29:45.070604 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.070526 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls podName:5f791f18-dd8e-46d2-8e38-cbdc9e26f75f nodeName:}" failed. No retries permitted until 2026-07-06 14:30:01.070510868 +0000 UTC m=+68.711480790 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls") pod "image-registry-9db9f9db4-nk4ht" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f") : secret "image-registry-tls" not found Jul 06 14:29:45.070604 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.070556 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls podName:c27dfc49-d472-4046-a1ac-55191f07e7c1 nodeName:}" failed. No retries permitted until 2026-07-06 14:30:01.070545991 +0000 UTC m=+68.711515906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls") pod "dns-default-czjlx" (UID: "c27dfc49-d472-4046-a1ac-55191f07e7c1") : secret "dns-default-metrics-tls" not found Jul 06 14:29:45.474090 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:45.474048 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:29:45.474571 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.474190 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:45.474571 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:45.474250 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:29:45.474571 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.474271 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls podName:fae66ff7-10e5-4713-a0a7-a2bfb0246aba nodeName:}" failed. No retries permitted until 2026-07-06 14:30:01.474254678 +0000 UTC m=+69.115224587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-vs5xn" (UID: "fae66ff7-10e5-4713-a0a7-a2bfb0246aba") : secret "cluster-monitoring-operator-tls" not found Jul 06 14:29:45.474571 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.474334 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:29:45.474571 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.474398 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls podName:7e898dfd-d2ea-40c1-8d51-2141622b10f7 nodeName:}" failed. No retries permitted until 2026-07-06 14:30:01.474385943 +0000 UTC m=+69.115355852 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-j8r5j" (UID: "7e898dfd-d2ea-40c1-8d51-2141622b10f7") : secret "samples-operator-tls" not found Jul 06 14:29:45.575176 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:45.575135 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:45.575365 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:45.575264 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:29:45.575365 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.575317 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 14:29:45.575365 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.575364 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:30:01.575351324 +0000 UTC m=+69.216321232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : configmap references non-existent config key: service-ca.crt Jul 06 14:29:45.575478 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:45.575379 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs podName:c56d1fb4-a9ab-41e7-b8ab-dd83500ec477 nodeName:}" failed. No retries permitted until 2026-07-06 14:30:01.575372883 +0000 UTC m=+69.216342792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs") pod "router-default-6fd947d8df-9l5bt" (UID: "c56d1fb4-a9ab-41e7-b8ab-dd83500ec477") : secret "router-metrics-certs-default" not found Jul 06 14:29:47.493253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:47.493220 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:47.493705 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:47.493304 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:47.493705 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:47.493331 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:29:47.493705 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:47.493401 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm podName:ea26ce84-0bf5-4578-a800-983f5646b2b7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:55.493384812 +0000 UTC m=+63.134354726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k5bxg" (UID: "ea26ce84-0bf5-4578-a800-983f5646b2b7") : configmap "kube-rbac-proxy" not found Jul 06 14:29:47.493705 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:47.493451 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:29:47.493705 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:47.493505 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls podName:ea26ce84-0bf5-4578-a800-983f5646b2b7 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:55.49349171 +0000 UTC m=+63.134461623 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5bxg" (UID: "ea26ce84-0bf5-4578-a800-983f5646b2b7") : secret "insights-runtime-extractor-tls" not found Jul 06 14:29:54.231687 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:54.231655 2569 scope.go:117] "RemoveContainer" containerID="3225dfb3bfd7235f377b76bb16b653319bc809b7420854111604f1bf0cbfc1b6" Jul 06 14:29:54.412086 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:54.412059 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q7ks8" Jul 06 14:29:54.536008 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:54.535983 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:29:54.536400 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:54.536383 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/1.log" Jul 06 14:29:54.536463 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:54.536420 2569 generic.go:356] "Generic (PLEG): container finished" podID="eafea937-7719-47d7-9240-e94089cdfcc4" containerID="f7c92123d132e70d1144dd489976b5bcb99ec113c2385340c6ad91231d9f0085" exitCode=255 Jul 06 14:29:54.536463 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:54.536448 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" event={"ID":"eafea937-7719-47d7-9240-e94089cdfcc4","Type":"ContainerDied","Data":"f7c92123d132e70d1144dd489976b5bcb99ec113c2385340c6ad91231d9f0085"} Jul 06 14:29:54.536528 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:54.536492 2569 scope.go:117] "RemoveContainer" containerID="3225dfb3bfd7235f377b76bb16b653319bc809b7420854111604f1bf0cbfc1b6" Jul 06 14:29:54.536817 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:54.536801 2569 scope.go:117] "RemoveContainer" containerID="f7c92123d132e70d1144dd489976b5bcb99ec113c2385340c6ad91231d9f0085" Jul 06 14:29:54.536986 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:54.536961 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-lsf6f_openshift-console-operator(eafea937-7719-47d7-9240-e94089cdfcc4)\"" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" podUID="eafea937-7719-47d7-9240-e94089cdfcc4" Jul 06 14:29:55.541361 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:55.541336 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:29:55.562618 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:55.562590 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:55.562743 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:55.562702 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:55.562784 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:55.562741 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:29:55.562818 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:55.562798 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls podName:ea26ce84-0bf5-4578-a800-983f5646b2b7 nodeName:}" failed. No retries permitted until 2026-07-06 14:30:11.562783274 +0000 UTC m=+79.203753187 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k5bxg" (UID: "ea26ce84-0bf5-4578-a800-983f5646b2b7") : secret "insights-runtime-extractor-tls" not found Jul 06 14:29:55.563161 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:55.563145 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ea26ce84-0bf5-4578-a800-983f5646b2b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:29:58.994388 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:58.994354 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:58.994783 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:58.994420 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:58.997328 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:58.997303 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 14:29:58.997533 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:58.997520 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 14:29:59.006926 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.006900 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/feefb1dd-3073-46f1-bd64-a43792da5c4f-metrics-certs\") pod \"network-metrics-daemon-wdpvc\" (UID: \"feefb1dd-3073-46f1-bd64-a43792da5c4f\") " pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:59.007028 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.006993 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e25332d-2afd-48fd-98a6-c42e1a6f93b9-original-pull-secret\") pod \"global-pull-secret-syncer-dzg6l\" (UID: \"2e25332d-2afd-48fd-98a6-c42e1a6f93b9\") " pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:59.046639 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.046614 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s774m\"" Jul 06 14:29:59.054937 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.054918 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wdpvc" Jul 06 14:29:59.060729 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.060713 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dzg6l" Jul 06 14:29:59.095282 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.095248 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lr2\" (UniqueName: \"kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2\") pod \"network-check-target-hc5qx\" (UID: \"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f\") " pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:59.098352 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.098300 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4lr2\" (UniqueName: \"kubernetes.io/projected/05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f-kube-api-access-m4lr2\") pod \"network-check-target-hc5qx\" (UID: \"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f\") " pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:59.189988 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.189935 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wdpvc"] Jul 06 14:29:59.191965 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:29:59.191936 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeefb1dd_3073_46f1_bd64_a43792da5c4f.slice/crio-076b7d9f33dc0d353978628476992833d4fe1771e64c567abc196e815f15b1b9 WatchSource:0}: Error finding container 076b7d9f33dc0d353978628476992833d4fe1771e64c567abc196e815f15b1b9: Status 404 returned error can't find the container with id 076b7d9f33dc0d353978628476992833d4fe1771e64c567abc196e815f15b1b9 Jul 06 14:29:59.206697 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.206677 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dzg6l"] Jul 06 14:29:59.209028 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:29:59.208999 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e25332d_2afd_48fd_98a6_c42e1a6f93b9.slice/crio-6915487ab24cfadd53e66415b20f265194147e5aa1594d0d10b2e418c89dab19 WatchSource:0}: Error finding container 6915487ab24cfadd53e66415b20f265194147e5aa1594d0d10b2e418c89dab19: Status 404 returned error can't find the container with id 6915487ab24cfadd53e66415b20f265194147e5aa1594d0d10b2e418c89dab19 Jul 06 14:29:59.249321 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.249265 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kwxdr\"" Jul 06 14:29:59.256281 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.256268 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:59.395676 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.395651 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hc5qx"] Jul 06 14:29:59.398057 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:29:59.398029 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05bf8c3f_4ed8_402c_bdb4_fe3e54b74d1f.slice/crio-63a76aa02573a7043906f3ef4f482daee4a97273a979541aadf3f75ef0efe6ce WatchSource:0}: Error finding container 63a76aa02573a7043906f3ef4f482daee4a97273a979541aadf3f75ef0efe6ce: Status 404 returned error can't find the container with id 63a76aa02573a7043906f3ef4f482daee4a97273a979541aadf3f75ef0efe6ce Jul 06 14:29:59.553956 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.553923 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dzg6l" event={"ID":"2e25332d-2afd-48fd-98a6-c42e1a6f93b9","Type":"ContainerStarted","Data":"6915487ab24cfadd53e66415b20f265194147e5aa1594d0d10b2e418c89dab19"} Jul 06 14:29:59.555258 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.555233 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hc5qx" event={"ID":"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f","Type":"ContainerStarted","Data":"aa7bc8cd393c4abaccf1aa7c185347da02a418712dfabc6bfc961729372144aa"} Jul 06 14:29:59.555359 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.555275 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hc5qx" event={"ID":"05bf8c3f-4ed8-402c-bdb4-fe3e54b74d1f","Type":"ContainerStarted","Data":"63a76aa02573a7043906f3ef4f482daee4a97273a979541aadf3f75ef0efe6ce"} Jul 06 14:29:59.555359 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.555350 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:29:59.556341 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.556314 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdpvc" event={"ID":"feefb1dd-3073-46f1-bd64-a43792da5c4f","Type":"ContainerStarted","Data":"076b7d9f33dc0d353978628476992833d4fe1771e64c567abc196e815f15b1b9"} Jul 06 14:29:59.573917 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.573879 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hc5qx" podStartSLOduration=65.573866136 podStartE2EDuration="1m5.573866136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:59.573297653 +0000 UTC m=+67.214267585" watchObservedRunningTime="2026-07-06 14:29:59.573866136 +0000 UTC m=+67.214836066" Jul 06 14:29:59.880264 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.880169 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:59.880264 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.880247 2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:29:59.880652 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:29:59.880636 2569 scope.go:117] "RemoveContainer" containerID="f7c92123d132e70d1144dd489976b5bcb99ec113c2385340c6ad91231d9f0085" Jul 06 14:29:59.880862 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:29:59.880822 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-lsf6f_openshift-console-operator(eafea937-7719-47d7-9240-e94089cdfcc4)\"" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" podUID="eafea937-7719-47d7-9240-e94089cdfcc4" Jul 06 14:30:01.116503 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.116414 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:30:01.116503 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.116473 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:30:01.118714 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.118694 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c27dfc49-d472-4046-a1ac-55191f07e7c1-metrics-tls\") pod \"dns-default-czjlx\" (UID: \"c27dfc49-d472-4046-a1ac-55191f07e7c1\") " pod="openshift-dns/dns-default-czjlx" Jul 06 14:30:01.118791 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.118746 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls\") pod \"image-registry-9db9f9db4-nk4ht\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:30:01.307907 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.307875 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vnmkm\"" Jul 06 14:30:01.315265 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.315233 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:30:01.330069 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.330043 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rs6zp\"" Jul 06 14:30:01.337658 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.337445 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-czjlx" Jul 06 14:30:01.479403 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.479373 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9db9f9db4-nk4ht"] Jul 06 14:30:01.482745 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:30:01.482712 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f791f18_dd8e_46d2_8e38_cbdc9e26f75f.slice/crio-780216f7bff93a9e7892e01381a0e580c5080379c682d59afb94d1d4f14861a4 WatchSource:0}: Error finding container 780216f7bff93a9e7892e01381a0e580c5080379c682d59afb94d1d4f14861a4: Status 404 returned error can't find the container with id 780216f7bff93a9e7892e01381a0e580c5080379c682d59afb94d1d4f14861a4 Jul 06 14:30:01.496173 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.496095 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-czjlx"] Jul 06 14:30:01.499116 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:30:01.499035 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc27dfc49_d472_4046_a1ac_55191f07e7c1.slice/crio-42d2f36c184c6e62ee9f9d5576771a56c348e66b0cc8437f89398693089bb15b WatchSource:0}: Error finding container 42d2f36c184c6e62ee9f9d5576771a56c348e66b0cc8437f89398693089bb15b: Status 404 returned error can't find the container with id 42d2f36c184c6e62ee9f9d5576771a56c348e66b0cc8437f89398693089bb15b Jul 06 14:30:01.520268 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.520230 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:30:01.520402 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.520343 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:30:01.522938 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.522912 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e898dfd-d2ea-40c1-8d51-2141622b10f7-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-j8r5j\" (UID: \"7e898dfd-d2ea-40c1-8d51-2141622b10f7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:30:01.524039 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.524016 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae66ff7-10e5-4713-a0a7-a2bfb0246aba-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-vs5xn\" (UID: \"fae66ff7-10e5-4713-a0a7-a2bfb0246aba\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:30:01.566221 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.566170 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-czjlx" event={"ID":"c27dfc49-d472-4046-a1ac-55191f07e7c1","Type":"ContainerStarted","Data":"42d2f36c184c6e62ee9f9d5576771a56c348e66b0cc8437f89398693089bb15b"} Jul 06 14:30:01.568652 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.568393 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdpvc" event={"ID":"feefb1dd-3073-46f1-bd64-a43792da5c4f","Type":"ContainerStarted","Data":"678f489fbd8dfc2fe9b3b53be0b9cf1933010a2dcb88d447f51ce70477f9bb4f"} Jul 06 14:30:01.568652 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.568456 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wdpvc" event={"ID":"feefb1dd-3073-46f1-bd64-a43792da5c4f","Type":"ContainerStarted","Data":"ebf2f3abe4c1204fcc3ddd5ff6392a57b3c12424a78fa39d17ccdd957a72c1c5"} Jul 06 14:30:01.570583 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.570557 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" event={"ID":"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f","Type":"ContainerStarted","Data":"3a07050b6b94eda8b68a7726bc05d474c3de55ed33ae1064bb060d67451cb815"} Jul 06 14:30:01.570711 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.570609 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:30:01.570711 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.570626 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" event={"ID":"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f","Type":"ContainerStarted","Data":"780216f7bff93a9e7892e01381a0e580c5080379c682d59afb94d1d4f14861a4"} Jul 06 14:30:01.595473 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.595410 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wdpvc" podStartSLOduration=65.93348942 podStartE2EDuration="1m7.595390602s" totalImagesPullingTime="1.661901182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:01.59418283 +0000 UTC m=+69.235152761" watchObservedRunningTime="2026-07-06 14:30:01.595390602 +0000 UTC m=+69.236360533" Jul 06 14:30:01.621770 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.621731 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:30:01.621937 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.621801 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:30:01.622432 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.622373 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-service-ca-bundle\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:30:01.624839 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.624815 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c56d1fb4-a9ab-41e7-b8ab-dd83500ec477-metrics-certs\") pod \"router-default-6fd947d8df-9l5bt\" (UID: \"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477\") " pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:30:01.624943 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.624850 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" podStartSLOduration=67.624830225 podStartE2EDuration="1m7.624830225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:01.624009664 +0000 UTC m=+69.264979621" watchObservedRunningTime="2026-07-06 14:30:01.624830225 +0000 UTC m=+69.265800159" Jul 06 14:30:01.664427 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.664395 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wdbkt\"" Jul 06 14:30:01.672951 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.672687 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" Jul 06 14:30:01.692806 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.692773 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jzn5z\"" Jul 06 14:30:01.701174 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.701134 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" Jul 06 14:30:01.776424 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.776395 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n2g9b\"" Jul 06 14:30:01.785694 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.785334 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:30:01.873116 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.872983 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j"] Jul 06 14:30:01.911328 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.911296 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn"] Jul 06 14:30:01.915018 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:30:01.914991 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae66ff7_10e5_4713_a0a7_a2bfb0246aba.slice/crio-90d6f2e3cabcaa6cae2605a5456a81ac5336915adcd5fd8fb2b1518774f23a6d WatchSource:0}: Error finding container 90d6f2e3cabcaa6cae2605a5456a81ac5336915adcd5fd8fb2b1518774f23a6d: Status 404 returned error can't find the container with id 90d6f2e3cabcaa6cae2605a5456a81ac5336915adcd5fd8fb2b1518774f23a6d Jul 06 14:30:01.980732 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:01.980697 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6fd947d8df-9l5bt"] Jul 06 14:30:01.985021 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:30:01.984992 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56d1fb4_a9ab_41e7_b8ab_dd83500ec477.slice/crio-7a0ab2941bc3f19782d399a9d3fd718016c3af711576c83d537b0468d5248f59 WatchSource:0}: Error finding container 7a0ab2941bc3f19782d399a9d3fd718016c3af711576c83d537b0468d5248f59: Status 404 returned error can't find the container with id 7a0ab2941bc3f19782d399a9d3fd718016c3af711576c83d537b0468d5248f59 Jul 06 14:30:02.576571 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:02.576531 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" event={"ID":"fae66ff7-10e5-4713-a0a7-a2bfb0246aba","Type":"ContainerStarted","Data":"90d6f2e3cabcaa6cae2605a5456a81ac5336915adcd5fd8fb2b1518774f23a6d"} Jul 06 14:30:02.578657 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:02.578405 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6fd947d8df-9l5bt" event={"ID":"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477","Type":"ContainerStarted","Data":"24a87b9abf62b04992520b09433e0232c765eff43ecb8f3a7ba551f49f9a27d6"} Jul 06 14:30:02.578657 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:02.578453 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6fd947d8df-9l5bt" event={"ID":"c56d1fb4-a9ab-41e7-b8ab-dd83500ec477","Type":"ContainerStarted","Data":"7a0ab2941bc3f19782d399a9d3fd718016c3af711576c83d537b0468d5248f59"} Jul 06 14:30:02.579780 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:02.579746 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" event={"ID":"7e898dfd-d2ea-40c1-8d51-2141622b10f7","Type":"ContainerStarted","Data":"29eeba26139988932a7956dbb6a51fc1aba7d61a731edf76a0ec78644a8c7d53"} Jul 06 14:30:02.613543 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:02.613481 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6fd947d8df-9l5bt" podStartSLOduration=33.613460584 podStartE2EDuration="33.613460584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:02.612922878 +0000 UTC m=+70.253892812" watchObservedRunningTime="2026-07-06 14:30:02.613460584 +0000 UTC m=+70.254430517" Jul 06 14:30:02.786783 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:02.786741 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:30:02.789994 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:02.789967 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:30:03.583399 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:03.583365 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:30:03.584884 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:03.584860 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6fd947d8df-9l5bt" Jul 06 14:30:04.593092 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:04.592920 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-czjlx" event={"ID":"c27dfc49-d472-4046-a1ac-55191f07e7c1","Type":"ContainerStarted","Data":"3509265ad218b1805c65976e4e05f8704219a14c30842e1737108128cbf8d8ef"} Jul 06 14:30:04.593092 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:04.593028 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-czjlx" Jul 06 14:30:04.593092 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:04.593056 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-czjlx" event={"ID":"c27dfc49-d472-4046-a1ac-55191f07e7c1","Type":"ContainerStarted","Data":"938e2d46c1e1a4cdc45ce4fc660e584eb271cd77961ab351d5575fd94f960fa1"} Jul 06 14:30:04.594501 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:04.594471 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dzg6l" event={"ID":"2e25332d-2afd-48fd-98a6-c42e1a6f93b9","Type":"ContainerStarted","Data":"276cf4352c9448a70a4849e5b225d032ce0b3307de3911cb4d58e780c0c6d0d8"} Jul 06 14:30:04.614642 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:04.614598 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-czjlx" podStartSLOduration=32.939055423 podStartE2EDuration="35.614584452s" totalImagesPullingTime="2.675529029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:04.613153942 +0000 UTC m=+72.254123874" watchObservedRunningTime="2026-07-06 14:30:04.614584452 +0000 UTC m=+72.255554382" Jul 06 14:30:04.634479 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:04.634424 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dzg6l" podStartSLOduration=65.66153844 podStartE2EDuration="1m10.63440536s" totalImagesPullingTime="4.97286692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:04.633344223 +0000 UTC m=+72.274314177" watchObservedRunningTime="2026-07-06 14:30:04.63440536 +0000 UTC m=+72.275375290" Jul 06 14:30:06.603540 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:06.603495 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" event={"ID":"7e898dfd-d2ea-40c1-8d51-2141622b10f7","Type":"ContainerStarted","Data":"4267b2c9863fab433a67a26e0dcf3ecf2e275b5a6d7eb36c047a822a8edbbaa2"} Jul 06 14:30:06.603989 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:06.603557 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" event={"ID":"7e898dfd-d2ea-40c1-8d51-2141622b10f7","Type":"ContainerStarted","Data":"c317406be3ffedce84e7a5d84c5874766b2835ea74de82c7bf26fb4cea3ab8fa"} Jul 06 14:30:06.604891 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:06.604869 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" event={"ID":"fae66ff7-10e5-4713-a0a7-a2bfb0246aba","Type":"ContainerStarted","Data":"cfb8e612c71dd47182ae89c9fd1b3335f3971de70376a5ef7884c35193558222"} Jul 06 14:30:06.626093 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:06.626043 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-j8r5j" podStartSLOduration=33.85682351 podStartE2EDuration="37.626029178s" totalImagesPullingTime="3.769205668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:06.625151475 +0000 UTC m=+74.266121407" watchObservedRunningTime="2026-07-06 14:30:06.626029178 +0000 UTC m=+74.266999108" Jul 06 14:30:06.646778 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:06.646736 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-vs5xn" podStartSLOduration=33.860726484 podStartE2EDuration="37.646722278s" totalImagesPullingTime="3.785995794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:06.646090593 +0000 UTC m=+74.287060524" watchObservedRunningTime="2026-07-06 14:30:06.646722278 +0000 UTC m=+74.287692250" Jul 06 14:30:11.396018 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.394702 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49"] Jul 06 14:30:11.424118 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.424086 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49"] Jul 06 14:30:11.424311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.424240 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.427310 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.427287 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 14:30:11.427418 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.427307 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 14:30:11.427418 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.427289 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ndt74\"" Jul 06 14:30:11.428394 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.428379 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 14:30:11.610950 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.610918 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:30:11.610950 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.610952 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/defedf5a-88aa-41a5-aba4-86e42864fe18-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.611190 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.610976 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/defedf5a-88aa-41a5-aba4-86e42864fe18-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.611190 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.610999 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg7v6\" (UniqueName: \"kubernetes.io/projected/defedf5a-88aa-41a5-aba4-86e42864fe18-kube-api-access-dg7v6\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.611190 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.611058 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/defedf5a-88aa-41a5-aba4-86e42864fe18-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.613273 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.613256 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ea26ce84-0bf5-4578-a800-983f5646b2b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k5bxg\" (UID: \"ea26ce84-0bf5-4578-a800-983f5646b2b7\") " pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:30:11.711796 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.711719 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/defedf5a-88aa-41a5-aba4-86e42864fe18-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.711796 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.711754 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/defedf5a-88aa-41a5-aba4-86e42864fe18-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.711796 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.711773 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg7v6\" (UniqueName: \"kubernetes.io/projected/defedf5a-88aa-41a5-aba4-86e42864fe18-kube-api-access-dg7v6\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.711796 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.711790 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/defedf5a-88aa-41a5-aba4-86e42864fe18-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.712493 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.712474 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/defedf5a-88aa-41a5-aba4-86e42864fe18-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.714144 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.714125 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/defedf5a-88aa-41a5-aba4-86e42864fe18-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.714438 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.714420 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/defedf5a-88aa-41a5-aba4-86e42864fe18-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.722100 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.722079 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg7v6\" (UniqueName: \"kubernetes.io/projected/defedf5a-88aa-41a5-aba4-86e42864fe18-kube-api-access-dg7v6\") pod \"prometheus-operator-567f7b8b7c-x7h49\" (UID: \"defedf5a-88aa-41a5-aba4-86e42864fe18\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.733901 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.733882 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" Jul 06 14:30:11.894837 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.894813 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49"] Jul 06 14:30:11.896735 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:30:11.896709 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddefedf5a_88aa_41a5_aba4_86e42864fe18.slice/crio-e322c66e97d24bd4fca2f2807aa7602d41f030ea4dd9e31364b5d584c572eb31 WatchSource:0}: Error finding container e322c66e97d24bd4fca2f2807aa7602d41f030ea4dd9e31364b5d584c572eb31: Status 404 returned error can't find the container with id e322c66e97d24bd4fca2f2807aa7602d41f030ea4dd9e31364b5d584c572eb31 Jul 06 14:30:11.901067 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.901048 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rzjwm\"" Jul 06 14:30:11.908329 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:11.908311 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k5bxg" Jul 06 14:30:12.067861 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:12.067839 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k5bxg"] Jul 06 14:30:12.072850 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:30:12.072822 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea26ce84_0bf5_4578_a800_983f5646b2b7.slice/crio-b12674773341afe4322e7c66c89fa33f517fa65757ade20e96eee5cd714b31de WatchSource:0}: Error finding container b12674773341afe4322e7c66c89fa33f517fa65757ade20e96eee5cd714b31de: Status 404 returned error can't find the container with id b12674773341afe4322e7c66c89fa33f517fa65757ade20e96eee5cd714b31de Jul 06 14:30:12.232020 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:12.231995 2569 scope.go:117] "RemoveContainer" containerID="f7c92123d132e70d1144dd489976b5bcb99ec113c2385340c6ad91231d9f0085" Jul 06 14:30:12.232188 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:30:12.232157 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-lsf6f_openshift-console-operator(eafea937-7719-47d7-9240-e94089cdfcc4)\"" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" podUID="eafea937-7719-47d7-9240-e94089cdfcc4" Jul 06 14:30:12.630134 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:12.630096 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" event={"ID":"defedf5a-88aa-41a5-aba4-86e42864fe18","Type":"ContainerStarted","Data":"e322c66e97d24bd4fca2f2807aa7602d41f030ea4dd9e31364b5d584c572eb31"} Jul 06 14:30:12.631654 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:12.631591 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k5bxg" event={"ID":"ea26ce84-0bf5-4578-a800-983f5646b2b7","Type":"ContainerStarted","Data":"b12674773341afe4322e7c66c89fa33f517fa65757ade20e96eee5cd714b31de"} Jul 06 14:30:14.601222 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:14.601170 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-czjlx" Jul 06 14:30:16.649248 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:16.649214 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k5bxg" event={"ID":"ea26ce84-0bf5-4578-a800-983f5646b2b7","Type":"ContainerStarted","Data":"c7324996fe348b94a401dacb446e00cce4c79c41e478e291849789293f9e1dff"} Jul 06 14:30:16.650952 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:16.650928 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" event={"ID":"defedf5a-88aa-41a5-aba4-86e42864fe18","Type":"ContainerStarted","Data":"f0ccee9ae67a3d88c0866c7bcc6a84827f8f5f44a2e6eca47ebb2cd936b306ce"} Jul 06 14:30:16.651073 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:16.650971 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" event={"ID":"defedf5a-88aa-41a5-aba4-86e42864fe18","Type":"ContainerStarted","Data":"54cd046afe1cf026d3906488b2eb9ca7dc75c2eeadb06937d98850132f35512f"} Jul 06 14:30:16.678026 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:16.677978 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x7h49" podStartSLOduration=1.491912323 podStartE2EDuration="5.677963289s" totalImagesPullingTime="4.186050966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:16.677199986 +0000 UTC m=+84.318169937" watchObservedRunningTime="2026-07-06 14:30:16.677963289 +0000 UTC m=+84.318933198" Jul 06 14:30:17.656701 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:17.656662 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k5bxg" event={"ID":"ea26ce84-0bf5-4578-a800-983f5646b2b7","Type":"ContainerStarted","Data":"374f263f8075ad8268f9c5ddbd1c321cfe29141889028add8975c4e86c44899c"} Jul 06 14:30:19.050916 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.050870 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7j2gj"] Jul 06 14:30:19.056398 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.056378 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.059588 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.059554 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 14:30:19.059588 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.059571 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 14:30:19.060117 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.060098 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 14:30:19.060687 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.060657 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-949sp\"" Jul 06 14:30:19.066122 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.066093 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7fb42739-96de-40b9-9ee4-1345292bc983-metrics-client-ca\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.066242 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.066149 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fb42739-96de-40b9-9ee4-1345292bc983-sys\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.066297 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.066241 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-accelerators-collector-config\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.066343 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.066294 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-wtmp\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.066392 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.066348 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7fb42739-96de-40b9-9ee4-1345292bc983-root\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.066392 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.066381 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.066490 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.066434 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-tls\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.066523 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.066488 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzv6l\" (UniqueName: \"kubernetes.io/projected/7fb42739-96de-40b9-9ee4-1345292bc983-kube-api-access-vzv6l\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.066523 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.066518 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-textfile\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.101269 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.101239 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc"] Jul 06 14:30:19.103958 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.103933 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.107789 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.107760 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 14:30:19.108296 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.108262 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 14:30:19.108415 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.108270 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8582q\"" Jul 06 14:30:19.108836 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.108815 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 14:30:19.123190 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.123143 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc"] Jul 06 14:30:19.166860 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.166833 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-tls\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167022 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.166868 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzv6l\" (UniqueName: \"kubernetes.io/projected/7fb42739-96de-40b9-9ee4-1345292bc983-kube-api-access-vzv6l\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167022 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.166895 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-textfile\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167022 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.166929 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7fb42739-96de-40b9-9ee4-1345292bc983-metrics-client-ca\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167022 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.166961 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fb42739-96de-40b9-9ee4-1345292bc983-sys\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167022 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.166996 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-accelerators-collector-config\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167306 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.167038 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-wtmp\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167306 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.167074 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7fb42739-96de-40b9-9ee4-1345292bc983-root\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167306 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.167104 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167306 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.167258 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fb42739-96de-40b9-9ee4-1345292bc983-sys\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167306 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.167271 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-textfile\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167306 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.167260 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7fb42739-96de-40b9-9ee4-1345292bc983-root\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167581 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.167330 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-wtmp\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167799 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.167781 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7fb42739-96de-40b9-9ee4-1345292bc983-metrics-client-ca\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.167876 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.167818 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-accelerators-collector-config\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.169575 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.169553 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.169660 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.169582 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7fb42739-96de-40b9-9ee4-1345292bc983-node-exporter-tls\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.189421 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.189397 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzv6l\" (UniqueName: \"kubernetes.io/projected/7fb42739-96de-40b9-9ee4-1345292bc983-kube-api-access-vzv6l\") pod \"node-exporter-7j2gj\" (UID: \"7fb42739-96de-40b9-9ee4-1345292bc983\") " pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.267731 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.267693 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.267893 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.267745 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/930f0621-c03a-4d4c-b3ad-22305faef6a1-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.267893 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.267856 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/930f0621-c03a-4d4c-b3ad-22305faef6a1-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.268006 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.267908 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scwm2\" (UniqueName: \"kubernetes.io/projected/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-api-access-scwm2\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.268072 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.268042 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.268136 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.268119 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.367053 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.366978 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7j2gj" Jul 06 14:30:19.369297 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.369270 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scwm2\" (UniqueName: \"kubernetes.io/projected/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-api-access-scwm2\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.369464 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.369316 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.369464 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.369375 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.369464 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.369413 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.369732 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:30:19.369593 2569 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 14:30:19.369732 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.369594 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/930f0621-c03a-4d4c-b3ad-22305faef6a1-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.369732 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:30:19.369657 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-tls podName:930f0621-c03a-4d4c-b3ad-22305faef6a1 nodeName:}" failed. No retries permitted until 2026-07-06 14:30:19.869637328 +0000 UTC m=+87.510607257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-4rwmc" (UID: "930f0621-c03a-4d4c-b3ad-22305faef6a1") : secret "kube-state-metrics-tls" not found Jul 06 14:30:19.369732 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.369712 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/930f0621-c03a-4d4c-b3ad-22305faef6a1-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.370153 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.370085 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/930f0621-c03a-4d4c-b3ad-22305faef6a1-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.370242 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.370172 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.370296 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.370260 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/930f0621-c03a-4d4c-b3ad-22305faef6a1-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.372195 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.372169 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.383661 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.383636 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scwm2\" (UniqueName: \"kubernetes.io/projected/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-api-access-scwm2\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.671601 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:30:19.671561 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb42739_96de_40b9_9ee4_1345292bc983.slice/crio-baa66b5ce552008ef85541d106160dd857aa257a62ca46405b1478d881c56ef6 WatchSource:0}: Error finding container baa66b5ce552008ef85541d106160dd857aa257a62ca46405b1478d881c56ef6: Status 404 returned error can't find the container with id baa66b5ce552008ef85541d106160dd857aa257a62ca46405b1478d881c56ef6 Jul 06 14:30:19.873244 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.873188 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:19.875597 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:19.875575 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/930f0621-c03a-4d4c-b3ad-22305faef6a1-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-4rwmc\" (UID: \"930f0621-c03a-4d4c-b3ad-22305faef6a1\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:20.013790 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:20.013706 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" Jul 06 14:30:20.162075 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:20.162046 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc"] Jul 06 14:30:20.163848 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:30:20.163818 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod930f0621_c03a_4d4c_b3ad_22305faef6a1.slice/crio-77e4344dd57d851eb97c0c2313af43bc03dae7710b0791a9708134376d729882 WatchSource:0}: Error finding container 77e4344dd57d851eb97c0c2313af43bc03dae7710b0791a9708134376d729882: Status 404 returned error can't find the container with id 77e4344dd57d851eb97c0c2313af43bc03dae7710b0791a9708134376d729882 Jul 06 14:30:20.672447 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:20.672411 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" event={"ID":"930f0621-c03a-4d4c-b3ad-22305faef6a1","Type":"ContainerStarted","Data":"77e4344dd57d851eb97c0c2313af43bc03dae7710b0791a9708134376d729882"} Jul 06 14:30:20.673739 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:20.673715 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7j2gj" event={"ID":"7fb42739-96de-40b9-9ee4-1345292bc983","Type":"ContainerStarted","Data":"baa66b5ce552008ef85541d106160dd857aa257a62ca46405b1478d881c56ef6"} Jul 06 14:30:20.675762 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:20.675743 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k5bxg" event={"ID":"ea26ce84-0bf5-4578-a800-983f5646b2b7","Type":"ContainerStarted","Data":"55c9aee60394e75dd2094350a6fbf6a769145b0ff14d94f76022172348edfc11"} Jul 06 14:30:20.703572 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:20.703528 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k5bxg" podStartSLOduration=34.195679029 podStartE2EDuration="41.703509987s" totalImagesPullingTime="7.507830958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:29:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:20.702784485 +0000 UTC m=+88.343754460" watchObservedRunningTime="2026-07-06 14:30:20.703509987 +0000 UTC m=+88.344479921" Jul 06 14:30:21.686096 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:21.686055 2569 generic.go:356] "Generic (PLEG): container finished" podID="7fb42739-96de-40b9-9ee4-1345292bc983" containerID="178750a517534f529a765754f6837546a72947ee39bb9c5169858644709afb74" exitCode=0 Jul 06 14:30:21.686544 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:21.686146 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7j2gj" event={"ID":"7fb42739-96de-40b9-9ee4-1345292bc983","Type":"ContainerDied","Data":"178750a517534f529a765754f6837546a72947ee39bb9c5169858644709afb74"} Jul 06 14:30:22.584584 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:22.584556 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:30:22.692576 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:22.692539 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" event={"ID":"930f0621-c03a-4d4c-b3ad-22305faef6a1","Type":"ContainerStarted","Data":"739004493ed124336f015e33980133aed7d0247483e4d8936fbf538b5dd39673"} Jul 06 14:30:22.692992 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:22.692600 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" event={"ID":"930f0621-c03a-4d4c-b3ad-22305faef6a1","Type":"ContainerStarted","Data":"a7f7bd3028da416a76d6c0e6ed2daed8442bf2f3d589495369ee7a2dd0fa0c30"} Jul 06 14:30:22.692992 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:22.692628 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" event={"ID":"930f0621-c03a-4d4c-b3ad-22305faef6a1","Type":"ContainerStarted","Data":"97123e36d6d33e717acfe977e678a994862e36c2dda58eb8822f74992c6d2829"} Jul 06 14:30:22.694675 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:22.694646 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7j2gj" event={"ID":"7fb42739-96de-40b9-9ee4-1345292bc983","Type":"ContainerStarted","Data":"1c556b72cfff8b5dd8d56a5023b03e96441a2e2682d8e13932df6fb0c61e1e15"} Jul 06 14:30:22.694777 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:22.694700 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7j2gj" event={"ID":"7fb42739-96de-40b9-9ee4-1345292bc983","Type":"ContainerStarted","Data":"2d2700008b7b38511825805861fc3b24482a39eeaacf9fef04a95bb934ee45be"} Jul 06 14:30:22.730646 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:22.730575 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-4rwmc" podStartSLOduration=2.194275375 podStartE2EDuration="3.730554725s" totalImagesPullingTime="1.53627935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:22.730075269 +0000 UTC m=+90.371045226" watchObservedRunningTime="2026-07-06 14:30:22.730554725 +0000 UTC m=+90.371524657" Jul 06 14:30:22.759233 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:22.759163 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7j2gj" podStartSLOduration=2.6964697060000002 podStartE2EDuration="3.759147408s" totalImagesPullingTime="992.719718ms" totalInitContainerRuntime="69.957984ms" isStatefulPod=true podCreationTimestamp="2026-07-06 14:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:22.757912556 +0000 UTC m=+90.398882488" watchObservedRunningTime="2026-07-06 14:30:22.759147408 +0000 UTC m=+90.400117340" Jul 06 14:30:23.607416 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.607381 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9db9f9db4-nk4ht"] Jul 06 14:30:23.713747 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.713714 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz"] Jul 06 14:30:23.716311 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.716288 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" Jul 06 14:30:23.719711 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.719692 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-fcxf7\"" Jul 06 14:30:23.719828 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.719796 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 14:30:23.742956 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.742933 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz"] Jul 06 14:30:23.803777 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.803737 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9238a25b-bdde-4d14-9051-5ec571e146c5-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-csswz\" (UID: \"9238a25b-bdde-4d14-9051-5ec571e146c5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" Jul 06 14:30:23.804104 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.804077 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv2rs\" (UniqueName: \"kubernetes.io/projected/9238a25b-bdde-4d14-9051-5ec571e146c5-kube-api-access-xv2rs\") pod \"monitoring-plugin-d5dcff7cf-csswz\" (UID: \"9238a25b-bdde-4d14-9051-5ec571e146c5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" Jul 06 14:30:23.904786 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.904703 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv2rs\" (UniqueName: \"kubernetes.io/projected/9238a25b-bdde-4d14-9051-5ec571e146c5-kube-api-access-xv2rs\") pod \"monitoring-plugin-d5dcff7cf-csswz\" (UID: \"9238a25b-bdde-4d14-9051-5ec571e146c5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" Jul 06 14:30:23.904786 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.904763 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9238a25b-bdde-4d14-9051-5ec571e146c5-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-csswz\" (UID: \"9238a25b-bdde-4d14-9051-5ec571e146c5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" Jul 06 14:30:23.907130 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.907098 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9238a25b-bdde-4d14-9051-5ec571e146c5-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-csswz\" (UID: \"9238a25b-bdde-4d14-9051-5ec571e146c5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" Jul 06 14:30:23.919239 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:23.919193 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv2rs\" (UniqueName: \"kubernetes.io/projected/9238a25b-bdde-4d14-9051-5ec571e146c5-kube-api-access-xv2rs\") pod \"monitoring-plugin-d5dcff7cf-csswz\" (UID: \"9238a25b-bdde-4d14-9051-5ec571e146c5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" Jul 06 14:30:24.025852 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:24.025819 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" Jul 06 14:30:24.169183 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:24.169160 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz"] Jul 06 14:30:24.171270 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:30:24.171239 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9238a25b_bdde_4d14_9051_5ec571e146c5.slice/crio-486bbbb3d0c33777a726e4349c8c972c9bbef3d87ea42f53155227a264307cd2 WatchSource:0}: Error finding container 486bbbb3d0c33777a726e4349c8c972c9bbef3d87ea42f53155227a264307cd2: Status 404 returned error can't find the container with id 486bbbb3d0c33777a726e4349c8c972c9bbef3d87ea42f53155227a264307cd2 Jul 06 14:30:24.708751 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:24.708716 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" event={"ID":"9238a25b-bdde-4d14-9051-5ec571e146c5","Type":"ContainerStarted","Data":"486bbbb3d0c33777a726e4349c8c972c9bbef3d87ea42f53155227a264307cd2"} Jul 06 14:30:25.232047 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:25.232017 2569 scope.go:117] "RemoveContainer" containerID="f7c92123d132e70d1144dd489976b5bcb99ec113c2385340c6ad91231d9f0085" Jul 06 14:30:25.715214 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:25.715173 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:30:25.715405 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:25.715307 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" event={"ID":"eafea937-7719-47d7-9240-e94089cdfcc4","Type":"ContainerStarted","Data":"9f1d56e52abbde3f7dd458f806df14c18e75bec02e7f3a45836d3f4f6823110c"} Jul 06 14:30:25.715672 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:25.715618 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:30:26.065770 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.065735 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" Jul 06 14:30:26.099725 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.099666 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-lsf6f" podStartSLOduration=49.532586034 podStartE2EDuration="57.099644991s" totalImagesPullingTime="7.567058957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:25.739682482 +0000 UTC m=+93.380652413" watchObservedRunningTime="2026-07-06 14:30:26.099644991 +0000 UTC m=+93.740614922" Jul 06 14:30:26.293968 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.293933 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-lbxkb"] Jul 06 14:30:26.297727 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.297700 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-lbxkb" Jul 06 14:30:26.301045 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.301024 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 14:30:26.301539 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.301518 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-w287b\"" Jul 06 14:30:26.301663 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.301585 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 14:30:26.318351 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.318296 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-lbxkb"] Jul 06 14:30:26.327645 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.327622 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lc9w\" (UniqueName: \"kubernetes.io/projected/e90e93c3-77b0-4cc6-99c6-6c0ecf20eff9-kube-api-access-8lc9w\") pod \"downloads-685bfb5f64-lbxkb\" (UID: \"e90e93c3-77b0-4cc6-99c6-6c0ecf20eff9\") " pod="openshift-console/downloads-685bfb5f64-lbxkb" Jul 06 14:30:26.428320 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.428278 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lc9w\" (UniqueName: \"kubernetes.io/projected/e90e93c3-77b0-4cc6-99c6-6c0ecf20eff9-kube-api-access-8lc9w\") pod \"downloads-685bfb5f64-lbxkb\" (UID: \"e90e93c3-77b0-4cc6-99c6-6c0ecf20eff9\") " pod="openshift-console/downloads-685bfb5f64-lbxkb" Jul 06 14:30:26.441363 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.441339 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lc9w\" (UniqueName: \"kubernetes.io/projected/e90e93c3-77b0-4cc6-99c6-6c0ecf20eff9-kube-api-access-8lc9w\") pod \"downloads-685bfb5f64-lbxkb\" (UID: \"e90e93c3-77b0-4cc6-99c6-6c0ecf20eff9\") " pod="openshift-console/downloads-685bfb5f64-lbxkb" Jul 06 14:30:26.607605 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.607528 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-lbxkb" Jul 06 14:30:26.727499 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.727461 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" event={"ID":"9238a25b-bdde-4d14-9051-5ec571e146c5","Type":"ContainerStarted","Data":"e998d2d56f9b25a1c541ae4fdb78cf295f8cd645c4a5da4b083f1c6f85b18278"} Jul 06 14:30:26.742851 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.742822 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-lbxkb"] Jul 06 14:30:26.746428 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:30:26.746401 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode90e93c3_77b0_4cc6_99c6_6c0ecf20eff9.slice/crio-923fb3ed382a2efd2e63d3cc0ae3a0a9c70ed00764e71f752cb663519fa69554 WatchSource:0}: Error finding container 923fb3ed382a2efd2e63d3cc0ae3a0a9c70ed00764e71f752cb663519fa69554: Status 404 returned error can't find the container with id 923fb3ed382a2efd2e63d3cc0ae3a0a9c70ed00764e71f752cb663519fa69554 Jul 06 14:30:26.749095 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:26.749044 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" podStartSLOduration=2.098510262 podStartE2EDuration="3.749028212s" totalImagesPullingTime="1.65051795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:30:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:26.748058356 +0000 UTC m=+94.389028286" watchObservedRunningTime="2026-07-06 14:30:26.749028212 +0000 UTC m=+94.389998157" Jul 06 14:30:27.734121 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:27.734070 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-lbxkb" event={"ID":"e90e93c3-77b0-4cc6-99c6-6c0ecf20eff9","Type":"ContainerStarted","Data":"923fb3ed382a2efd2e63d3cc0ae3a0a9c70ed00764e71f752cb663519fa69554"} Jul 06 14:30:27.734772 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:27.734709 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" Jul 06 14:30:27.740810 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:27.740760 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-csswz" Jul 06 14:30:30.563182 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:30.563112 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hc5qx" Jul 06 14:30:47.818639 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:47.818592 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-lbxkb" event={"ID":"e90e93c3-77b0-4cc6-99c6-6c0ecf20eff9","Type":"ContainerStarted","Data":"ad3a37ee834e2dde0885b85d78bda1339817134ac45bf140feaf4c98d201e557"} Jul 06 14:30:47.819195 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:47.818819 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-lbxkb" Jul 06 14:30:47.840223 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:47.840174 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-lbxkb" Jul 06 14:30:47.845166 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:47.845107 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-lbxkb" podStartSLOduration=1.599046746 podStartE2EDuration="21.845089556s" totalImagesPullingTime="20.24604281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:30:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:47.842400635 +0000 UTC m=+115.483370607" watchObservedRunningTime="2026-07-06 14:30:47.845089556 +0000 UTC m=+115.486059492" Jul 06 14:30:48.627829 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:48.627732 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" podUID="5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" containerName="registry" containerID="cri-o://3a07050b6b94eda8b68a7726bc05d474c3de55ed33ae1064bb060d67451cb815" gracePeriod=30 Jul 06 14:30:48.827844 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:48.827804 2569 generic.go:356] "Generic (PLEG): container finished" podID="5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" containerID="3a07050b6b94eda8b68a7726bc05d474c3de55ed33ae1064bb060d67451cb815" exitCode=0 Jul 06 14:30:48.829778 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:48.827938 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" event={"ID":"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f","Type":"ContainerDied","Data":"3a07050b6b94eda8b68a7726bc05d474c3de55ed33ae1064bb060d67451cb815"} Jul 06 14:30:48.829778 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:48.829576 2569 generic.go:356] "Generic (PLEG): container finished" podID="17cfe1b2-63bb-4b47-b4c9-82f9ff59c333" containerID="81e5113786915d5479bcf5b2b87e9e2cc7098491b3a5109ff6616e5d5e5e88e4" exitCode=0 Jul 06 14:30:48.829778 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:48.829639 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" event={"ID":"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333","Type":"ContainerDied","Data":"81e5113786915d5479bcf5b2b87e9e2cc7098491b3a5109ff6616e5d5e5e88e4"} Jul 06 14:30:48.830137 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:48.830114 2569 scope.go:117] "RemoveContainer" containerID="81e5113786915d5479bcf5b2b87e9e2cc7098491b3a5109ff6616e5d5e5e88e4" Jul 06 14:30:48.945440 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:48.945416 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:30:49.039928 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.039890 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fq2m\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-kube-api-access-4fq2m\") pod \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " Jul 06 14:30:49.040116 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.039950 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-bound-sa-token\") pod \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " Jul 06 14:30:49.040116 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.039988 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-image-registry-private-configuration\") pod \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " Jul 06 14:30:49.040116 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.040038 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-trusted-ca\") pod \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " Jul 06 14:30:49.040116 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.040074 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-ca-trust-extracted\") pod \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " Jul 06 14:30:49.040380 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.040227 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls\") pod \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " Jul 06 14:30:49.040380 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.040290 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-installation-pull-secrets\") pod \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " Jul 06 14:30:49.040380 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.040324 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-certificates\") pod \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\" (UID: \"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f\") " Jul 06 14:30:49.040905 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.040528 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:30:49.040905 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.040641 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-trusted-ca\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:30:49.040905 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.040869 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:30:49.045144 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.043722 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:30:49.045144 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.045096 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:30:49.045687 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.045645 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:30:49.045972 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.045941 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-kube-api-access-4fq2m" (OuterVolumeSpecName: "kube-api-access-4fq2m") pod "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f"). InnerVolumeSpecName "kube-api-access-4fq2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:30:49.047869 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.047844 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:30:49.052197 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.052172 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" (UID: "5f791f18-dd8e-46d2-8e38-cbdc9e26f75f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:30:49.141316 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.141234 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-tls\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:30:49.141316 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.141267 2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-installation-pull-secrets\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:30:49.141316 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.141283 2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-registry-certificates\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:30:49.141316 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.141298 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fq2m\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-kube-api-access-4fq2m\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:30:49.141316 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.141312 2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-bound-sa-token\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:30:49.141675 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.141327 2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-image-registry-private-configuration\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:30:49.141675 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.141342 2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f-ca-trust-extracted\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:30:49.836154 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.835589 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" Jul 06 14:30:49.836154 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.835619 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9db9f9db4-nk4ht" event={"ID":"5f791f18-dd8e-46d2-8e38-cbdc9e26f75f","Type":"ContainerDied","Data":"780216f7bff93a9e7892e01381a0e580c5080379c682d59afb94d1d4f14861a4"} Jul 06 14:30:49.836154 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.835690 2569 scope.go:117] "RemoveContainer" containerID="3a07050b6b94eda8b68a7726bc05d474c3de55ed33ae1064bb060d67451cb815" Jul 06 14:30:49.838488 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.838457 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-mqcxb" event={"ID":"17cfe1b2-63bb-4b47-b4c9-82f9ff59c333","Type":"ContainerStarted","Data":"0155ed6b598209ae5741925f68b48b32dd4b35bfcb89f6fae92492d04186fd9c"} Jul 06 14:30:49.885080 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.885043 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9db9f9db4-nk4ht"] Jul 06 14:30:49.899648 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:49.899613 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9db9f9db4-nk4ht"] Jul 06 14:30:51.237510 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:30:51.237478 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" path="/var/lib/kubelet/pods/5f791f18-dd8e-46d2-8e38-cbdc9e26f75f/volumes" Jul 06 14:31:03.898563 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:31:03.898533 2569 generic.go:356] "Generic (PLEG): container finished" podID="86455cea-7f7b-47d6-a140-2809930847c8" containerID="9ed81b8877800a36dc64bf967d660e59b7731d47a3d44582d089bd2fb2023ce7" exitCode=0 Jul 06 14:31:03.898963 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:31:03.898606 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" event={"ID":"86455cea-7f7b-47d6-a140-2809930847c8","Type":"ContainerDied","Data":"9ed81b8877800a36dc64bf967d660e59b7731d47a3d44582d089bd2fb2023ce7"} Jul 06 14:31:03.899047 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:31:03.899029 2569 scope.go:117] "RemoveContainer" containerID="9ed81b8877800a36dc64bf967d660e59b7731d47a3d44582d089bd2fb2023ce7" Jul 06 14:31:04.903933 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:31:04.903896 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5nw2p" event={"ID":"86455cea-7f7b-47d6-a140-2809930847c8","Type":"ContainerStarted","Data":"a9cb3cd5bc371c8ea4814ccd7a79d19495bf4e5cda9517da2f804acf4e9a93d0"} Jul 06 14:31:08.919274 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:31:08.919172 2569 generic.go:356] "Generic (PLEG): container finished" podID="5fb1a5f8-6190-4002-9c4f-a36897910ac3" containerID="86c04038b5446cd7db42e2abe4574b40aebcadc97d3061477e1afbeb41324bb0" exitCode=0 Jul 06 14:31:08.919274 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:31:08.919251 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" event={"ID":"5fb1a5f8-6190-4002-9c4f-a36897910ac3","Type":"ContainerDied","Data":"86c04038b5446cd7db42e2abe4574b40aebcadc97d3061477e1afbeb41324bb0"} Jul 06 14:31:08.919655 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:31:08.919589 2569 scope.go:117] "RemoveContainer" containerID="86c04038b5446cd7db42e2abe4574b40aebcadc97d3061477e1afbeb41324bb0" Jul 06 14:31:09.924405 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:31:09.924373 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-l2xzw" event={"ID":"5fb1a5f8-6190-4002-9c4f-a36897910ac3","Type":"ContainerStarted","Data":"80cecf7220d8afd615264df4eec24ff12f315d4e5b4a3db611f19a88b4b584bd"} Jul 06 14:33:53.137869 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:33:53.137831 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:33:53.141575 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:33:53.141551 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:33:53.142115 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:33:53.142093 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:33:53.153033 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:33:53.153012 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:33:53.154685 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:33:53.154654 2569 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 14:34:16.013414 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.013378 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd"] Jul 06 14:34:16.014737 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.013691 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" containerName="registry" Jul 06 14:34:16.014737 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.013703 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" containerName="registry" Jul 06 14:34:16.014737 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.013766 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f791f18-dd8e-46d2-8e38-cbdc9e26f75f" containerName="registry" Jul 06 14:34:16.015682 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.015662 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:16.018233 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.018215 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-n4mpl\"" Jul 06 14:34:16.018575 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.018555 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 14:34:16.019176 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.019163 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 14:34:16.025431 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.025408 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd"] Jul 06 14:34:16.132390 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.132347 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqb2m\" (UniqueName: \"kubernetes.io/projected/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-kube-api-access-wqb2m\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:16.132390 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.132396 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:16.132631 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.132452 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:16.233427 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.233395 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:16.233427 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.233433 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:16.233610 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.233506 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqb2m\" (UniqueName: \"kubernetes.io/projected/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-kube-api-access-wqb2m\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:16.233831 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.233809 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:16.233867 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.233838 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:16.243104 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.243079 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqb2m\" (UniqueName: \"kubernetes.io/projected/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-kube-api-access-wqb2m\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:16.325058 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.325035 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:16.454986 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.454959 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd"] Jul 06 14:34:16.457094 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:34:16.457066 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b8320c_1be0_4fc5_88f1_5c9feb2c7334.slice/crio-3d8947160cbc504614208d400892d2442568b8a677c26b5f8eaa068a22e16adf WatchSource:0}: Error finding container 3d8947160cbc504614208d400892d2442568b8a677c26b5f8eaa068a22e16adf: Status 404 returned error can't find the container with id 3d8947160cbc504614208d400892d2442568b8a677c26b5f8eaa068a22e16adf Jul 06 14:34:16.459077 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.459056 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:34:16.614851 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:16.614758 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" event={"ID":"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334","Type":"ContainerStarted","Data":"3d8947160cbc504614208d400892d2442568b8a677c26b5f8eaa068a22e16adf"} Jul 06 14:34:25.655802 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:25.655766 2569 generic.go:356] "Generic (PLEG): container finished" podID="c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" containerID="bfa5245e65eea8b85918dcf86d9a16de88c3682f528882d12eebdc0e76427f07" exitCode=0 Jul 06 14:34:25.656248 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:25.655817 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" event={"ID":"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334","Type":"ContainerDied","Data":"bfa5245e65eea8b85918dcf86d9a16de88c3682f528882d12eebdc0e76427f07"} Jul 06 14:34:27.666804 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:27.666718 2569 generic.go:356] "Generic (PLEG): container finished" podID="c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" containerID="c96111e4474096233fdffea6d796e2ec98b6f8e06d18c7e939ffedc40210794e" exitCode=0 Jul 06 14:34:27.666804 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:27.666792 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" event={"ID":"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334","Type":"ContainerDied","Data":"c96111e4474096233fdffea6d796e2ec98b6f8e06d18c7e939ffedc40210794e"} Jul 06 14:34:36.705547 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:36.705457 2569 generic.go:356] "Generic (PLEG): container finished" podID="c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" containerID="b6d9580b7d9c84975282d0a55683b620b83e52f2a500fba11447f8bceb3f067c" exitCode=0 Jul 06 14:34:36.705547 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:36.705514 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" event={"ID":"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334","Type":"ContainerDied","Data":"b6d9580b7d9c84975282d0a55683b620b83e52f2a500fba11447f8bceb3f067c"} Jul 06 14:34:37.841134 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:37.841105 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:38.011725 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.011638 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqb2m\" (UniqueName: \"kubernetes.io/projected/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-kube-api-access-wqb2m\") pod \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " Jul 06 14:34:38.011725 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.011703 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-bundle\") pod \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " Jul 06 14:34:38.011912 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.011745 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-util\") pod \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\" (UID: \"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334\") " Jul 06 14:34:38.012402 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.012374 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-bundle" (OuterVolumeSpecName: "bundle") pod "c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" (UID: "c9b8320c-1be0-4fc5-88f1-5c9feb2c7334"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:34:38.013901 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.013874 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-kube-api-access-wqb2m" (OuterVolumeSpecName: "kube-api-access-wqb2m") pod "c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" (UID: "c9b8320c-1be0-4fc5-88f1-5c9feb2c7334"). InnerVolumeSpecName "kube-api-access-wqb2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:34:38.016011 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.015990 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-util" (OuterVolumeSpecName: "util") pod "c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" (UID: "c9b8320c-1be0-4fc5-88f1-5c9feb2c7334"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:34:38.112431 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.112397 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-util\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:34:38.112431 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.112426 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqb2m\" (UniqueName: \"kubernetes.io/projected/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-kube-api-access-wqb2m\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:34:38.112431 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.112437 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9b8320c-1be0-4fc5-88f1-5c9feb2c7334-bundle\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:34:38.715444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.715406 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" event={"ID":"c9b8320c-1be0-4fc5-88f1-5c9feb2c7334","Type":"ContainerDied","Data":"3d8947160cbc504614208d400892d2442568b8a677c26b5f8eaa068a22e16adf"} Jul 06 14:34:38.715444 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.715449 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754084wdd" Jul 06 14:34:38.715657 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:38.715455 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d8947160cbc504614208d400892d2442568b8a677c26b5f8eaa068a22e16adf" Jul 06 14:34:44.451648 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.451610 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m"] Jul 06 14:34:44.452253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.451938 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" containerName="extract" Jul 06 14:34:44.452253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.451956 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" containerName="extract" Jul 06 14:34:44.452253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.451973 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" containerName="util" Jul 06 14:34:44.452253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.451981 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" containerName="util" Jul 06 14:34:44.452253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.452008 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" containerName="pull" Jul 06 14:34:44.452253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.452016 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" containerName="pull" Jul 06 14:34:44.452253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.452106 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9b8320c-1be0-4fc5-88f1-5c9feb2c7334" containerName="extract" Jul 06 14:34:44.455580 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.455561 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m" Jul 06 14:34:44.458132 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.458116 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 14:34:44.458257 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.458118 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7r95j\"" Jul 06 14:34:44.458257 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.458118 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:34:44.466975 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.466954 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m"] Jul 06 14:34:44.563307 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.563255 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szwvx\" (UniqueName: \"kubernetes.io/projected/18c435bd-459c-4471-82cf-261597cea3fd-kube-api-access-szwvx\") pod \"cert-manager-operator-controller-manager-55c444bf86-jp87m\" (UID: \"18c435bd-459c-4471-82cf-261597cea3fd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m" Jul 06 14:34:44.563485 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.563344 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18c435bd-459c-4471-82cf-261597cea3fd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jp87m\" (UID: \"18c435bd-459c-4471-82cf-261597cea3fd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m" Jul 06 14:34:44.664335 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.664306 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szwvx\" (UniqueName: \"kubernetes.io/projected/18c435bd-459c-4471-82cf-261597cea3fd-kube-api-access-szwvx\") pod \"cert-manager-operator-controller-manager-55c444bf86-jp87m\" (UID: \"18c435bd-459c-4471-82cf-261597cea3fd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m" Jul 06 14:34:44.664494 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.664353 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18c435bd-459c-4471-82cf-261597cea3fd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jp87m\" (UID: \"18c435bd-459c-4471-82cf-261597cea3fd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m" Jul 06 14:34:44.664666 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.664651 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18c435bd-459c-4471-82cf-261597cea3fd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jp87m\" (UID: \"18c435bd-459c-4471-82cf-261597cea3fd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m" Jul 06 14:34:44.675171 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.675140 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szwvx\" (UniqueName: \"kubernetes.io/projected/18c435bd-459c-4471-82cf-261597cea3fd-kube-api-access-szwvx\") pod \"cert-manager-operator-controller-manager-55c444bf86-jp87m\" (UID: \"18c435bd-459c-4471-82cf-261597cea3fd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m" Jul 06 14:34:44.765170 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.765149 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m" Jul 06 14:34:44.896834 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:44.896682 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m"] Jul 06 14:34:44.899309 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:34:44.899270 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c435bd_459c_4471_82cf_261597cea3fd.slice/crio-3a8c96c30413c88b22de7d16d282e8e51c8fcf4cee6bcaa16d35cfdd59d4605c WatchSource:0}: Error finding container 3a8c96c30413c88b22de7d16d282e8e51c8fcf4cee6bcaa16d35cfdd59d4605c: Status 404 returned error can't find the container with id 3a8c96c30413c88b22de7d16d282e8e51c8fcf4cee6bcaa16d35cfdd59d4605c Jul 06 14:34:45.743279 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:45.743238 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m" event={"ID":"18c435bd-459c-4471-82cf-261597cea3fd","Type":"ContainerStarted","Data":"3a8c96c30413c88b22de7d16d282e8e51c8fcf4cee6bcaa16d35cfdd59d4605c"} Jul 06 14:34:46.749432 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:46.749388 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m" event={"ID":"18c435bd-459c-4471-82cf-261597cea3fd","Type":"ContainerStarted","Data":"663ff5534aae62fc021936031b8cba3b77f3ca009dc30882f75dce2bbe31479b"} Jul 06 14:34:46.806162 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:46.806105 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jp87m" podStartSLOduration=1.119772485 podStartE2EDuration="2.806089205s" totalImagesPullingTime="1.68631672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:34:46.805932485 +0000 UTC m=+354.446902417" watchObservedRunningTime="2026-07-06 14:34:46.806089205 +0000 UTC m=+354.447059136" Jul 06 14:34:49.519944 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.519900 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8lgnq"] Jul 06 14:34:49.523812 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.523792 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" Jul 06 14:34:49.526489 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.526459 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-wc4nl\"" Jul 06 14:34:49.527700 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.527683 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 14:34:49.527769 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.527738 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 14:34:49.531918 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.531897 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8lgnq"] Jul 06 14:34:49.607492 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.607457 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76gjb\" (UniqueName: \"kubernetes.io/projected/d5027f16-fe99-4fd8-8bbb-5e718633fd1a-kube-api-access-76gjb\") pod \"cert-manager-webhook-6486ffd997-8lgnq\" (UID: \"d5027f16-fe99-4fd8-8bbb-5e718633fd1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" Jul 06 14:34:49.607668 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.607512 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5027f16-fe99-4fd8-8bbb-5e718633fd1a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8lgnq\" (UID: \"d5027f16-fe99-4fd8-8bbb-5e718633fd1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" Jul 06 14:34:49.708357 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.708320 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76gjb\" (UniqueName: \"kubernetes.io/projected/d5027f16-fe99-4fd8-8bbb-5e718633fd1a-kube-api-access-76gjb\") pod \"cert-manager-webhook-6486ffd997-8lgnq\" (UID: \"d5027f16-fe99-4fd8-8bbb-5e718633fd1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" Jul 06 14:34:49.708555 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.708386 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5027f16-fe99-4fd8-8bbb-5e718633fd1a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8lgnq\" (UID: \"d5027f16-fe99-4fd8-8bbb-5e718633fd1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" Jul 06 14:34:49.720731 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.720695 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5027f16-fe99-4fd8-8bbb-5e718633fd1a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8lgnq\" (UID: \"d5027f16-fe99-4fd8-8bbb-5e718633fd1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" Jul 06 14:34:49.720849 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.720807 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76gjb\" (UniqueName: \"kubernetes.io/projected/d5027f16-fe99-4fd8-8bbb-5e718633fd1a-kube-api-access-76gjb\") pod \"cert-manager-webhook-6486ffd997-8lgnq\" (UID: \"d5027f16-fe99-4fd8-8bbb-5e718633fd1a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" Jul 06 14:34:49.847305 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.847201 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" Jul 06 14:34:49.976329 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:49.976200 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8lgnq"] Jul 06 14:34:49.978600 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:34:49.978567 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5027f16_fe99_4fd8_8bbb_5e718633fd1a.slice/crio-8a34e6c91b872e485ba4cffb7827b2d6facc0c4098be57d0db4f644ccd18778c WatchSource:0}: Error finding container 8a34e6c91b872e485ba4cffb7827b2d6facc0c4098be57d0db4f644ccd18778c: Status 404 returned error can't find the container with id 8a34e6c91b872e485ba4cffb7827b2d6facc0c4098be57d0db4f644ccd18778c Jul 06 14:34:50.768767 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:50.768728 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" event={"ID":"d5027f16-fe99-4fd8-8bbb-5e718633fd1a","Type":"ContainerStarted","Data":"8a34e6c91b872e485ba4cffb7827b2d6facc0c4098be57d0db4f644ccd18778c"} Jul 06 14:34:51.161889 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.161813 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn"] Jul 06 14:34:51.164095 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.164078 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn" Jul 06 14:34:51.166405 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.166382 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-cv4t4\"" Jul 06 14:34:51.175316 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.175290 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn"] Jul 06 14:34:51.219578 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.219542 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccfcd433-46ef-4ca9-b107-4d1113670c7c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vw6zn\" (UID: \"ccfcd433-46ef-4ca9-b107-4d1113670c7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn" Jul 06 14:34:51.219724 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.219607 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6v4k\" (UniqueName: \"kubernetes.io/projected/ccfcd433-46ef-4ca9-b107-4d1113670c7c-kube-api-access-r6v4k\") pod \"cert-manager-cainjector-559cdd8c87-vw6zn\" (UID: \"ccfcd433-46ef-4ca9-b107-4d1113670c7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn" Jul 06 14:34:51.320173 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.320137 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6v4k\" (UniqueName: \"kubernetes.io/projected/ccfcd433-46ef-4ca9-b107-4d1113670c7c-kube-api-access-r6v4k\") pod \"cert-manager-cainjector-559cdd8c87-vw6zn\" (UID: \"ccfcd433-46ef-4ca9-b107-4d1113670c7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn" Jul 06 14:34:51.320401 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.320230 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccfcd433-46ef-4ca9-b107-4d1113670c7c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vw6zn\" (UID: \"ccfcd433-46ef-4ca9-b107-4d1113670c7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn" Jul 06 14:34:51.330187 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.330159 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccfcd433-46ef-4ca9-b107-4d1113670c7c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vw6zn\" (UID: \"ccfcd433-46ef-4ca9-b107-4d1113670c7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn" Jul 06 14:34:51.330473 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.330454 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6v4k\" (UniqueName: \"kubernetes.io/projected/ccfcd433-46ef-4ca9-b107-4d1113670c7c-kube-api-access-r6v4k\") pod \"cert-manager-cainjector-559cdd8c87-vw6zn\" (UID: \"ccfcd433-46ef-4ca9-b107-4d1113670c7c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn" Jul 06 14:34:51.473559 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.473484 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn" Jul 06 14:34:51.600647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.600622 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn"] Jul 06 14:34:51.603064 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:34:51.603037 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podccfcd433_46ef_4ca9_b107_4d1113670c7c.slice/crio-481926807c9ca4cce3bbfd40503f269474e869e251b9f3ca701298f2b1b46a20 WatchSource:0}: Error finding container 481926807c9ca4cce3bbfd40503f269474e869e251b9f3ca701298f2b1b46a20: Status 404 returned error can't find the container with id 481926807c9ca4cce3bbfd40503f269474e869e251b9f3ca701298f2b1b46a20 Jul 06 14:34:51.775329 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:51.775291 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn" event={"ID":"ccfcd433-46ef-4ca9-b107-4d1113670c7c","Type":"ContainerStarted","Data":"481926807c9ca4cce3bbfd40503f269474e869e251b9f3ca701298f2b1b46a20"} Jul 06 14:34:53.787615 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:53.787577 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" event={"ID":"d5027f16-fe99-4fd8-8bbb-5e718633fd1a","Type":"ContainerStarted","Data":"95c54fcee6cb603bccf236f66f8bd9814e2ff5f2d308514be90a44e13ae998cd"} Jul 06 14:34:53.788071 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:53.787786 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" Jul 06 14:34:53.789654 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:53.789628 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn" event={"ID":"ccfcd433-46ef-4ca9-b107-4d1113670c7c","Type":"ContainerStarted","Data":"c9e3285c01829589131ec437ed417a3a1c432a657c4fef4100628b5eb6d89164"} Jul 06 14:34:53.814890 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:53.814832 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" podStartSLOduration=1.105267935 podStartE2EDuration="4.814811309s" totalImagesPullingTime="3.709543374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:34:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:34:53.812862222 +0000 UTC m=+361.453832183" watchObservedRunningTime="2026-07-06 14:34:53.814811309 +0000 UTC m=+361.455781241" Jul 06 14:34:53.833509 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:53.833451 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vw6zn" podStartSLOduration=0.738592529 podStartE2EDuration="2.833436205s" totalImagesPullingTime="2.094843676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:34:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:34:53.832450313 +0000 UTC m=+361.473420245" watchObservedRunningTime="2026-07-06 14:34:53.833436205 +0000 UTC m=+361.474406135" Jul 06 14:34:59.796836 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:34:59.796803 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8lgnq" Jul 06 14:35:23.363805 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.363769 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h"] Jul 06 14:35:23.366470 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.366452 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:23.369332 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.369310 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-n4mpl\"" Jul 06 14:35:23.369440 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.369311 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 14:35:23.369440 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.369379 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 14:35:23.377458 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.377437 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h"] Jul 06 14:35:23.477950 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.477923 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:23.478125 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.477961 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:23.478125 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.478007 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svth5\" (UniqueName: \"kubernetes.io/projected/890f960d-de3f-491f-8769-5aded2744e3b-kube-api-access-svth5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:23.578503 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.578471 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svth5\" (UniqueName: \"kubernetes.io/projected/890f960d-de3f-491f-8769-5aded2744e3b-kube-api-access-svth5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:23.578674 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.578524 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:23.578674 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.578550 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:23.578903 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.578886 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:23.578982 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.578927 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:23.588753 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.588727 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svth5\" (UniqueName: \"kubernetes.io/projected/890f960d-de3f-491f-8769-5aded2744e3b-kube-api-access-svth5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:23.675476 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.675395 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:23.810176 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.810146 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h"] Jul 06 14:35:23.811961 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:35:23.811935 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod890f960d_de3f_491f_8769_5aded2744e3b.slice/crio-38f4fcdbd5ba6247e94034adf03d5d77db73f9de68b00084172d8535e6e329be WatchSource:0}: Error finding container 38f4fcdbd5ba6247e94034adf03d5d77db73f9de68b00084172d8535e6e329be: Status 404 returned error can't find the container with id 38f4fcdbd5ba6247e94034adf03d5d77db73f9de68b00084172d8535e6e329be Jul 06 14:35:23.926541 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.926445 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" event={"ID":"890f960d-de3f-491f-8769-5aded2744e3b","Type":"ContainerStarted","Data":"1dea74339e460e52bba667311600f01d449506b84fdca8ee7a8c32eef1a1603b"} Jul 06 14:35:23.926541 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:23.926527 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" event={"ID":"890f960d-de3f-491f-8769-5aded2744e3b","Type":"ContainerStarted","Data":"38f4fcdbd5ba6247e94034adf03d5d77db73f9de68b00084172d8535e6e329be"} Jul 06 14:35:24.931777 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:24.931691 2569 generic.go:356] "Generic (PLEG): container finished" podID="890f960d-de3f-491f-8769-5aded2744e3b" containerID="1dea74339e460e52bba667311600f01d449506b84fdca8ee7a8c32eef1a1603b" exitCode=0 Jul 06 14:35:24.932125 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:24.931777 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" event={"ID":"890f960d-de3f-491f-8769-5aded2744e3b","Type":"ContainerDied","Data":"1dea74339e460e52bba667311600f01d449506b84fdca8ee7a8c32eef1a1603b"} Jul 06 14:35:25.938022 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:25.937932 2569 generic.go:356] "Generic (PLEG): container finished" podID="890f960d-de3f-491f-8769-5aded2744e3b" containerID="393348daa6a89968c9cf97372a4817887aa0bbb282f11e5140fa895a8fe277de" exitCode=0 Jul 06 14:35:25.938451 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:25.938016 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" event={"ID":"890f960d-de3f-491f-8769-5aded2744e3b","Type":"ContainerDied","Data":"393348daa6a89968c9cf97372a4817887aa0bbb282f11e5140fa895a8fe277de"} Jul 06 14:35:26.945155 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:26.945117 2569 generic.go:356] "Generic (PLEG): container finished" podID="890f960d-de3f-491f-8769-5aded2744e3b" containerID="cdc6bbbd655487ad5c013c1f357a5e1b37d0380cad774c36bcdbef5e6f52f51f" exitCode=0 Jul 06 14:35:26.945578 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:26.945222 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" event={"ID":"890f960d-de3f-491f-8769-5aded2744e3b","Type":"ContainerDied","Data":"cdc6bbbd655487ad5c013c1f357a5e1b37d0380cad774c36bcdbef5e6f52f51f"} Jul 06 14:35:28.069877 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.069855 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:28.218732 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.218655 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-util\") pod \"890f960d-de3f-491f-8769-5aded2744e3b\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " Jul 06 14:35:28.218732 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.218697 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-bundle\") pod \"890f960d-de3f-491f-8769-5aded2744e3b\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " Jul 06 14:35:28.218920 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.218735 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svth5\" (UniqueName: \"kubernetes.io/projected/890f960d-de3f-491f-8769-5aded2744e3b-kube-api-access-svth5\") pod \"890f960d-de3f-491f-8769-5aded2744e3b\" (UID: \"890f960d-de3f-491f-8769-5aded2744e3b\") " Jul 06 14:35:28.219437 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.219405 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-bundle" (OuterVolumeSpecName: "bundle") pod "890f960d-de3f-491f-8769-5aded2744e3b" (UID: "890f960d-de3f-491f-8769-5aded2744e3b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:35:28.220762 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.220738 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/890f960d-de3f-491f-8769-5aded2744e3b-kube-api-access-svth5" (OuterVolumeSpecName: "kube-api-access-svth5") pod "890f960d-de3f-491f-8769-5aded2744e3b" (UID: "890f960d-de3f-491f-8769-5aded2744e3b"). InnerVolumeSpecName "kube-api-access-svth5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:35:28.226968 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.226942 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-util" (OuterVolumeSpecName: "util") pod "890f960d-de3f-491f-8769-5aded2744e3b" (UID: "890f960d-de3f-491f-8769-5aded2744e3b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:35:28.319953 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.319916 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-util\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:35:28.319953 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.319948 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/890f960d-de3f-491f-8769-5aded2744e3b-bundle\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:35:28.320154 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.319961 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svth5\" (UniqueName: \"kubernetes.io/projected/890f960d-de3f-491f-8769-5aded2744e3b-kube-api-access-svth5\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:35:28.956253 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.956191 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" Jul 06 14:35:28.956496 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.956190 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftcv8h" event={"ID":"890f960d-de3f-491f-8769-5aded2744e3b","Type":"ContainerDied","Data":"38f4fcdbd5ba6247e94034adf03d5d77db73f9de68b00084172d8535e6e329be"} Jul 06 14:35:28.956496 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:28.956379 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38f4fcdbd5ba6247e94034adf03d5d77db73f9de68b00084172d8535e6e329be" Jul 06 14:35:36.227696 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.227654 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg"] Jul 06 14:35:36.228088 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.227987 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="890f960d-de3f-491f-8769-5aded2744e3b" containerName="pull" Jul 06 14:35:36.228088 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.228000 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="890f960d-de3f-491f-8769-5aded2744e3b" containerName="pull" Jul 06 14:35:36.228088 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.228009 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="890f960d-de3f-491f-8769-5aded2744e3b" containerName="extract" Jul 06 14:35:36.228088 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.228014 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="890f960d-de3f-491f-8769-5aded2744e3b" containerName="extract" Jul 06 14:35:36.228088 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.228022 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="890f960d-de3f-491f-8769-5aded2744e3b" containerName="util" Jul 06 14:35:36.228088 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.228027 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="890f960d-de3f-491f-8769-5aded2744e3b" containerName="util" Jul 06 14:35:36.228088 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.228084 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="890f960d-de3f-491f-8769-5aded2744e3b" containerName="extract" Jul 06 14:35:36.230344 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.230329 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:36.235193 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:35:36.235168 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"webhook-server-cert\" is forbidden: User \"system:node:ip-10-0-137-102.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-102.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" type="*v1.Secret" Jul 06 14:35:36.236803 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:35:36.236775 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"lws-manager-config\" is forbidden: User \"system:node:ip-10-0-137-102.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-102.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" type="*v1.ConfigMap" Jul 06 14:35:36.236873 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:35:36.236775 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-102.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-102.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 06 14:35:36.238296 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:35:36.238259 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-102.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-102.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 06 14:35:36.238296 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:35:36.238280 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-cert\" is forbidden: User \"system:node:ip-10-0-137-102.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-102.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" type="*v1.Secret" Jul 06 14:35:36.239227 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:35:36.239177 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"lws-controller-manager-6689bc4cf6-h5mtg\" is forbidden: User \"system:node:ip-10-0-137-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-102.ec2.internal' and this object" podUID="825a13a3-1d3b-44af-b557-59d5b20d0d98" pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:36.240351 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:35:36.240325 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"lws-controller-manager-dockercfg-ctvhf\" is forbidden: User \"system:node:ip-10-0-137-102.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-137-102.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ctvhf\"" type="*v1.Secret" Jul 06 14:35:36.259869 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.259845 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg"] Jul 06 14:35:36.388288 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.388250 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfxx5\" (UniqueName: \"kubernetes.io/projected/825a13a3-1d3b-44af-b557-59d5b20d0d98-kube-api-access-qfxx5\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:36.388467 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.388361 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/825a13a3-1d3b-44af-b557-59d5b20d0d98-manager-config\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:36.388467 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.388454 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/825a13a3-1d3b-44af-b557-59d5b20d0d98-cert\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:36.388586 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.388487 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/825a13a3-1d3b-44af-b557-59d5b20d0d98-metrics-cert\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:36.489502 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.489413 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfxx5\" (UniqueName: \"kubernetes.io/projected/825a13a3-1d3b-44af-b557-59d5b20d0d98-kube-api-access-qfxx5\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:36.489649 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.489499 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/825a13a3-1d3b-44af-b557-59d5b20d0d98-manager-config\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:36.489649 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.489554 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/825a13a3-1d3b-44af-b557-59d5b20d0d98-cert\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:36.489649 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:36.489586 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/825a13a3-1d3b-44af-b557-59d5b20d0d98-metrics-cert\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:37.118468 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.118432 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 14:35:37.122781 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.122756 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/825a13a3-1d3b-44af-b557-59d5b20d0d98-metrics-cert\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:37.153416 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.153390 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 14:35:37.160476 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.160456 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/825a13a3-1d3b-44af-b557-59d5b20d0d98-manager-config\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:37.238294 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.238264 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ctvhf\"" Jul 06 14:35:37.442024 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.441934 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 14:35:37.467919 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.467888 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 14:35:37.472328 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.472302 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/825a13a3-1d3b-44af-b557-59d5b20d0d98-cert\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:37.580105 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.580073 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:35:37.591882 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.591856 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfxx5\" (UniqueName: \"kubernetes.io/projected/825a13a3-1d3b-44af-b557-59d5b20d0d98-kube-api-access-qfxx5\") pod \"lws-controller-manager-6689bc4cf6-h5mtg\" (UID: \"825a13a3-1d3b-44af-b557-59d5b20d0d98\") " pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:37.738792 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.738670 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:37.887742 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.887718 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg"] Jul 06 14:35:37.890058 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:35:37.890032 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825a13a3_1d3b_44af_b557_59d5b20d0d98.slice/crio-dddb665593d84842855f8d2a88206a2c85058e28c665b9a2b0339ae57d5e5e63 WatchSource:0}: Error finding container dddb665593d84842855f8d2a88206a2c85058e28c665b9a2b0339ae57d5e5e63: Status 404 returned error can't find the container with id dddb665593d84842855f8d2a88206a2c85058e28c665b9a2b0339ae57d5e5e63 Jul 06 14:35:37.995161 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:37.995067 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" event={"ID":"825a13a3-1d3b-44af-b557-59d5b20d0d98","Type":"ContainerStarted","Data":"dddb665593d84842855f8d2a88206a2c85058e28c665b9a2b0339ae57d5e5e63"} Jul 06 14:35:41.013384 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.013351 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" event={"ID":"825a13a3-1d3b-44af-b557-59d5b20d0d98","Type":"ContainerStarted","Data":"67bf2b1975fb7294be19e9daeaaa90469aecd0fbb7e752916e7cd25b8453a3e4"} Jul 06 14:35:41.013777 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.013477 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:41.039132 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.039071 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" podStartSLOduration=2.701878484 podStartE2EDuration="5.039050627s" totalImagesPullingTime="2.337172143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:35:41.038138521 +0000 UTC m=+408.679108452" watchObservedRunningTime="2026-07-06 14:35:41.039050627 +0000 UTC m=+408.680020558" Jul 06 14:35:41.236179 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.236147 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm"] Jul 06 14:35:41.239152 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.239126 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.248141 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.248120 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 14:35:41.248442 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.248425 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-r5xql\"" Jul 06 14:35:41.249038 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.249022 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 14:35:41.249080 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.249033 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 14:35:41.256632 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.256611 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 14:35:41.276497 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.276471 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm"] Jul 06 14:35:41.328849 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.328811 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.329047 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.328960 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-tmp\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.329047 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.329008 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-manifests\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.329047 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.329034 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-webhook-cert\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.329169 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.329100 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvczm\" (UniqueName: \"kubernetes.io/projected/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-kube-api-access-qvczm\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.429511 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.429477 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvczm\" (UniqueName: \"kubernetes.io/projected/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-kube-api-access-qvczm\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.429688 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.429523 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.429688 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.429565 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-tmp\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.429688 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.429589 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-manifests\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.429688 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.429603 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-webhook-cert\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.432666 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.430367 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-manifests\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.436870 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.433059 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-webhook-cert\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.437066 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.437044 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-tmp\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.437796 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.437777 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.461950 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.461922 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvczm\" (UniqueName: \"kubernetes.io/projected/2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05-kube-api-access-qvczm\") pod \"opendatahub-operator-controller-manager-75545b5d88-4swgm\" (UID: \"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05\") " pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.548530 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.548440 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:41.685081 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:41.685055 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm"] Jul 06 14:35:41.687165 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:35:41.687131 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b34ec27_1b0b_45d8_a7f9_bb3490a3dc05.slice/crio-8e28a347e10d4dd3735e17e97447346a2ba89789cb3006916b76e38cedcd1c49 WatchSource:0}: Error finding container 8e28a347e10d4dd3735e17e97447346a2ba89789cb3006916b76e38cedcd1c49: Status 404 returned error can't find the container with id 8e28a347e10d4dd3735e17e97447346a2ba89789cb3006916b76e38cedcd1c49 Jul 06 14:35:42.018937 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:42.018900 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" event={"ID":"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05","Type":"ContainerStarted","Data":"8e28a347e10d4dd3735e17e97447346a2ba89789cb3006916b76e38cedcd1c49"} Jul 06 14:35:45.036555 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:45.036518 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" event={"ID":"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05","Type":"ContainerStarted","Data":"251f0bdd24e09218def33118c9b154f4975f36c47f206cf8863b61f17ff23197"} Jul 06 14:35:46.041943 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:46.041906 2569 generic.go:356] "Generic (PLEG): container finished" podID="2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05" containerID="251f0bdd24e09218def33118c9b154f4975f36c47f206cf8863b61f17ff23197" exitCode=0 Jul 06 14:35:46.042404 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:46.041976 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" event={"ID":"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05","Type":"ContainerDied","Data":"251f0bdd24e09218def33118c9b154f4975f36c47f206cf8863b61f17ff23197"} Jul 06 14:35:47.047979 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:47.047938 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" event={"ID":"2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05","Type":"ContainerStarted","Data":"140958beca95454ef93b49c488dd9fb424cc4f75d0bc39d489e3c55a7acd8e96"} Jul 06 14:35:47.048407 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:47.048051 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:35:47.118830 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:47.118780 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" podStartSLOduration=1.593500665 podStartE2EDuration="6.118763664s" totalImagesPullingTime="3.465852722s" totalInitContainerRuntime="1.059410277s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:35:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:35:47.116809727 +0000 UTC m=+414.757779668" watchObservedRunningTime="2026-07-06 14:35:47.118763664 +0000 UTC m=+414.759733595" Jul 06 14:35:52.020943 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:52.020913 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6689bc4cf6-h5mtg" Jul 06 14:35:58.053742 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:35:58.053713 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-75545b5d88-4swgm" Jul 06 14:36:14.056785 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.056748 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6"] Jul 06 14:36:14.067903 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.067876 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.077657 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.077637 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 14:36:14.077770 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.077698 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 14:36:14.077910 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.077890 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rs6qh\"" Jul 06 14:36:14.078031 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.078015 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 14:36:14.079643 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.079628 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 14:36:14.100150 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.100121 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6"] Jul 06 14:36:14.181473 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.181442 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.181473 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.181471 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv2m6\" (UniqueName: \"kubernetes.io/projected/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-kube-api-access-cv2m6\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.181677 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.181493 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.181677 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.181579 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.181677 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.181606 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.181677 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.181635 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.181830 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.181693 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.282158 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.282122 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.282158 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.282159 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.282440 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.282198 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.282440 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.282267 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.282440 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.282305 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.282440 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.282328 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv2m6\" (UniqueName: \"kubernetes.io/projected/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-kube-api-access-cv2m6\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.282440 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.282354 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.282909 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.282859 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.284831 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.284800 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.284949 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.284929 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.285186 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.285161 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.285186 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.285175 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.293119 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.293095 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv2m6\" (UniqueName: \"kubernetes.io/projected/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-kube-api-access-cv2m6\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.295634 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.295614 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xzsp6\" (UID: \"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.377123 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.377049 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:14.532190 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:14.532163 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6"] Jul 06 14:36:14.534668 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:36:14.534639 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab8e800_bbb8_4b4b_baca_c8ebf85d90c3.slice/crio-badebd2738f98c37519ded7a33b2ae00dc9c42a28d696fd7896198d0f3a3c22d WatchSource:0}: Error finding container badebd2738f98c37519ded7a33b2ae00dc9c42a28d696fd7896198d0f3a3c22d: Status 404 returned error can't find the container with id badebd2738f98c37519ded7a33b2ae00dc9c42a28d696fd7896198d0f3a3c22d Jul 06 14:36:15.175894 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:15.175856 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" event={"ID":"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3","Type":"ContainerStarted","Data":"badebd2738f98c37519ded7a33b2ae00dc9c42a28d696fd7896198d0f3a3c22d"} Jul 06 14:36:17.239448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:17.239400 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 14:36:17.239839 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:17.239472 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 14:36:18.192639 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:18.192604 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" event={"ID":"3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3","Type":"ContainerStarted","Data":"04f941d4aac6987aef4b0addd1f92c87a23cb972ba02aa6ac58b06aa905d49ee"} Jul 06 14:36:18.192799 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:18.192737 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:36:18.194342 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:18.194316 2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-xzsp6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:36:18.194445 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:18.194373 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" podUID="3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:36:18.232622 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:18.232576 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" podStartSLOduration=2.530074124 podStartE2EDuration="5.232561955s" totalImagesPullingTime="2.702487831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:36:18.22936218 +0000 UTC m=+445.870332111" watchObservedRunningTime="2026-07-06 14:36:18.232561955 +0000 UTC m=+445.873531887" Jul 06 14:36:19.197882 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:36:19.197853 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xzsp6" Jul 06 14:38:53.058932 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.058890 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:38:53.062002 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.061987 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" Jul 06 14:38:53.065962 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.065940 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 14:38:53.066990 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.066966 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 14:38:53.067097 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.067007 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 14:38:53.067097 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.066978 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t92s5\"" Jul 06 14:38:53.079211 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.079186 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:38:53.124500 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.124469 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e1e49d62-0ca4-4f2c-9698-f7a450b97bb6-config-file\") pod \"limitador-limitador-54d6d988d5-htz88\" (UID: \"e1e49d62-0ca4-4f2c-9698-f7a450b97bb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" Jul 06 14:38:53.124500 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.124501 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfx8p\" (UniqueName: \"kubernetes.io/projected/e1e49d62-0ca4-4f2c-9698-f7a450b97bb6-kube-api-access-zfx8p\") pod \"limitador-limitador-54d6d988d5-htz88\" (UID: \"e1e49d62-0ca4-4f2c-9698-f7a450b97bb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" Jul 06 14:38:53.135287 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.135258 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:38:53.176804 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.176778 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:38:53.177703 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.177684 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:38:53.180979 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.180956 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:38:53.181689 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.181670 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:38:53.225739 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.225716 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e1e49d62-0ca4-4f2c-9698-f7a450b97bb6-config-file\") pod \"limitador-limitador-54d6d988d5-htz88\" (UID: \"e1e49d62-0ca4-4f2c-9698-f7a450b97bb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" Jul 06 14:38:53.225855 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.225742 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfx8p\" (UniqueName: \"kubernetes.io/projected/e1e49d62-0ca4-4f2c-9698-f7a450b97bb6-kube-api-access-zfx8p\") pod \"limitador-limitador-54d6d988d5-htz88\" (UID: \"e1e49d62-0ca4-4f2c-9698-f7a450b97bb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" Jul 06 14:38:53.229385 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.229370 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 14:38:53.236449 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.236428 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e1e49d62-0ca4-4f2c-9698-f7a450b97bb6-config-file\") pod \"limitador-limitador-54d6d988d5-htz88\" (UID: \"e1e49d62-0ca4-4f2c-9698-f7a450b97bb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" Jul 06 14:38:53.238298 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.238280 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 14:38:53.248927 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.248908 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 14:38:53.258007 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.257988 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfx8p\" (UniqueName: \"kubernetes.io/projected/e1e49d62-0ca4-4f2c-9698-f7a450b97bb6-kube-api-access-zfx8p\") pod \"limitador-limitador-54d6d988d5-htz88\" (UID: \"e1e49d62-0ca4-4f2c-9698-f7a450b97bb6\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" Jul 06 14:38:53.375278 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.375195 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t92s5\"" Jul 06 14:38:53.383588 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.383570 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" Jul 06 14:38:53.522695 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.522670 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:38:53.524853 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:38:53.524825 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e49d62_0ca4_4f2c_9698_f7a450b97bb6.slice/crio-7650f1dbb39305885291ffaf087d97e8d5891fa69d46adf339acf6f3d4899815 WatchSource:0}: Error finding container 7650f1dbb39305885291ffaf087d97e8d5891fa69d46adf339acf6f3d4899815: Status 404 returned error can't find the container with id 7650f1dbb39305885291ffaf087d97e8d5891fa69d46adf339acf6f3d4899815 Jul 06 14:38:53.881668 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:53.881629 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" event={"ID":"e1e49d62-0ca4-4f2c-9698-f7a450b97bb6","Type":"ContainerStarted","Data":"7650f1dbb39305885291ffaf087d97e8d5891fa69d46adf339acf6f3d4899815"} Jul 06 14:38:56.901462 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:56.901419 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" event={"ID":"e1e49d62-0ca4-4f2c-9698-f7a450b97bb6","Type":"ContainerStarted","Data":"8e5471b8b34b1712930edf1064301657deac3eed7795dc6ee3bd8eb81eb275e7"} Jul 06 14:38:56.901911 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:56.901490 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" Jul 06 14:38:56.924852 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:38:56.924803 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" podStartSLOduration=1.37581678 podStartE2EDuration="3.924788337s" totalImagesPullingTime="2.548971557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:38:56.923076559 +0000 UTC m=+604.564046502" watchObservedRunningTime="2026-07-06 14:38:56.924788337 +0000 UTC m=+604.565758268" Jul 06 14:39:07.907325 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:39:07.907287 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-htz88" Jul 06 14:40:03.084984 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:40:03.084942 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:40:22.926930 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:40:22.926891 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:40:45.430506 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:40:45.430419 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:40:51.926000 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:40:51.925968 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:41:06.744178 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:06.744134 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:41:14.024059 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.024022 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq"] Jul 06 14:41:14.027381 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.027362 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.029936 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.029915 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 14:41:14.030378 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.030348 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 14:41:14.031164 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.031150 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-q94n4\"" Jul 06 14:41:14.031316 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.031184 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 14:41:14.037837 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.037810 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq"] Jul 06 14:41:14.118507 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.118467 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vvf9\" (UniqueName: \"kubernetes.io/projected/62fe3ea9-bb50-4159-a278-c834a118a64b-kube-api-access-9vvf9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.118507 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.118510 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.118712 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.118535 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.118712 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.118557 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.118712 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.118595 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.118712 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.118665 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62fe3ea9-bb50-4159-a278-c834a118a64b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.219776 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.219725 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62fe3ea9-bb50-4159-a278-c834a118a64b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.219965 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.219796 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vvf9\" (UniqueName: \"kubernetes.io/projected/62fe3ea9-bb50-4159-a278-c834a118a64b-kube-api-access-9vvf9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.219965 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.219833 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.219965 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.219856 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.219965 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.219880 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.220167 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.219983 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.220329 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.220298 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.220450 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.220331 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.220450 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.220408 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.222148 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.222126 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62fe3ea9-bb50-4159-a278-c834a118a64b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.222345 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.222327 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62fe3ea9-bb50-4159-a278-c834a118a64b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.230366 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.230340 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vvf9\" (UniqueName: \"kubernetes.io/projected/62fe3ea9-bb50-4159-a278-c834a118a64b-kube-api-access-9vvf9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w9ssq\" (UID: \"62fe3ea9-bb50-4159-a278-c834a118a64b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.338677 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.338580 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:14.472427 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.472397 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq"] Jul 06 14:41:14.473693 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:41:14.473669 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62fe3ea9_bb50_4159_a278_c834a118a64b.slice/crio-c31ac18e85700e17851618e25aa95002a33f35b49d0400b78b84776dafa6387e WatchSource:0}: Error finding container c31ac18e85700e17851618e25aa95002a33f35b49d0400b78b84776dafa6387e: Status 404 returned error can't find the container with id c31ac18e85700e17851618e25aa95002a33f35b49d0400b78b84776dafa6387e Jul 06 14:41:14.474990 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.474973 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:41:14.534260 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:14.534227 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" event={"ID":"62fe3ea9-bb50-4159-a278-c834a118a64b","Type":"ContainerStarted","Data":"c31ac18e85700e17851618e25aa95002a33f35b49d0400b78b84776dafa6387e"} Jul 06 14:41:16.227851 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:16.227818 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:41:16.546069 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:16.546034 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" event={"ID":"62fe3ea9-bb50-4159-a278-c834a118a64b","Type":"ContainerStarted","Data":"1936ce9a41484f6fd9676e095ab0701c9f626e3f28ab355445b968795fa0394b"} Jul 06 14:41:16.568167 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:16.568115 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" podStartSLOduration=1.475804356 podStartE2EDuration="2.568100092s" totalImagesPullingTime="1.092295736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:41:16.566661529 +0000 UTC m=+744.207631485" watchObservedRunningTime="2026-07-06 14:41:16.568100092 +0000 UTC m=+744.209070022" Jul 06 14:41:23.137224 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:23.137170 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:41:24.339132 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:24.339093 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:24.339514 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:24.339172 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:24.351721 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:24.351697 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:41:24.593736 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:41:24.593664 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w9ssq" Jul 06 14:42:37.545889 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:42:37.545846 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:42:47.048315 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:42:47.048270 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:42:58.720096 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:42:58.720057 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:43:07.537083 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:43:07.537048 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:43:17.655660 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:43:17.655620 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:43:45.534734 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:43:45.534649 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:43:53.206096 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:43:53.206067 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:43:53.206711 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:43:53.206695 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:43:53.210487 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:43:53.210463 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:43:53.210897 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:43:53.210880 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:43:56.432077 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:43:56.432018 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:44:34.826312 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:44:34.826277 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:44:44.025339 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:44:44.025301 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:44:54.236932 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:44:54.236890 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:45:00.162441 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:00.162395 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722485-f45qj"] Jul 06 14:45:00.165943 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:00.165919 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" Jul 06 14:45:00.169927 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:00.169900 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2zlpp\"" Jul 06 14:45:00.177864 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:00.177832 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722485-f45qj"] Jul 06 14:45:00.262751 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:00.262707 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmtgk\" (UniqueName: \"kubernetes.io/projected/adf65a14-adcd-42a7-8ea8-ba0ed4977a02-kube-api-access-gmtgk\") pod \"maas-api-key-cleanup-29722485-f45qj\" (UID: \"adf65a14-adcd-42a7-8ea8-ba0ed4977a02\") " pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" Jul 06 14:45:00.363681 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:00.363628 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmtgk\" (UniqueName: \"kubernetes.io/projected/adf65a14-adcd-42a7-8ea8-ba0ed4977a02-kube-api-access-gmtgk\") pod \"maas-api-key-cleanup-29722485-f45qj\" (UID: \"adf65a14-adcd-42a7-8ea8-ba0ed4977a02\") " pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" Jul 06 14:45:00.375026 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:00.374989 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmtgk\" (UniqueName: \"kubernetes.io/projected/adf65a14-adcd-42a7-8ea8-ba0ed4977a02-kube-api-access-gmtgk\") pod \"maas-api-key-cleanup-29722485-f45qj\" (UID: \"adf65a14-adcd-42a7-8ea8-ba0ed4977a02\") " pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" Jul 06 14:45:00.477863 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:00.477761 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" Jul 06 14:45:00.631883 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:00.631855 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722485-f45qj"] Jul 06 14:45:00.633893 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:45:00.633860 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadf65a14_adcd_42a7_8ea8_ba0ed4977a02.slice/crio-45ba5abface4d71436b7119fce2cd1a1aee3972c5909bfbc992e907cb2e2b6a1 WatchSource:0}: Error finding container 45ba5abface4d71436b7119fce2cd1a1aee3972c5909bfbc992e907cb2e2b6a1: Status 404 returned error can't find the container with id 45ba5abface4d71436b7119fce2cd1a1aee3972c5909bfbc992e907cb2e2b6a1 Jul 06 14:45:01.614142 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:01.614103 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" event={"ID":"adf65a14-adcd-42a7-8ea8-ba0ed4977a02","Type":"ContainerStarted","Data":"45ba5abface4d71436b7119fce2cd1a1aee3972c5909bfbc992e907cb2e2b6a1"} Jul 06 14:45:03.320577 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:03.320533 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:45:03.627199 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:03.627160 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" event={"ID":"adf65a14-adcd-42a7-8ea8-ba0ed4977a02","Type":"ContainerStarted","Data":"ccd7e720b6094eaaf768bcf066f557ad9467c89bc69c5435a819fc714ef97bcc"} Jul 06 14:45:03.654976 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:03.654921 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" podStartSLOduration=1.125012086 podStartE2EDuration="3.654902051s" totalImagesPullingTime="2.529889965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:45:03.653224151 +0000 UTC m=+971.294194073" watchObservedRunningTime="2026-07-06 14:45:03.654902051 +0000 UTC m=+971.295871982" Jul 06 14:45:04.633840 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:04.633805 2569 generic.go:356] "Generic (PLEG): container finished" podID="adf65a14-adcd-42a7-8ea8-ba0ed4977a02" containerID="ccd7e720b6094eaaf768bcf066f557ad9467c89bc69c5435a819fc714ef97bcc" exitCode=0 Jul 06 14:45:04.634333 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:04.633848 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" event={"ID":"adf65a14-adcd-42a7-8ea8-ba0ed4977a02","Type":"ContainerDied","Data":"ccd7e720b6094eaaf768bcf066f557ad9467c89bc69c5435a819fc714ef97bcc"} Jul 06 14:45:05.783841 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:05.783807 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" Jul 06 14:45:05.809284 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:05.809250 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmtgk\" (UniqueName: \"kubernetes.io/projected/adf65a14-adcd-42a7-8ea8-ba0ed4977a02-kube-api-access-gmtgk\") pod \"adf65a14-adcd-42a7-8ea8-ba0ed4977a02\" (UID: \"adf65a14-adcd-42a7-8ea8-ba0ed4977a02\") " Jul 06 14:45:05.812041 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:05.812008 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adf65a14-adcd-42a7-8ea8-ba0ed4977a02-kube-api-access-gmtgk" (OuterVolumeSpecName: "kube-api-access-gmtgk") pod "adf65a14-adcd-42a7-8ea8-ba0ed4977a02" (UID: "adf65a14-adcd-42a7-8ea8-ba0ed4977a02"). InnerVolumeSpecName "kube-api-access-gmtgk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:45:05.910905 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:05.910815 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmtgk\" (UniqueName: \"kubernetes.io/projected/adf65a14-adcd-42a7-8ea8-ba0ed4977a02-kube-api-access-gmtgk\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:45:06.644301 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:06.644272 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" Jul 06 14:45:06.644301 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:06.644282 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722485-f45qj" event={"ID":"adf65a14-adcd-42a7-8ea8-ba0ed4977a02","Type":"ContainerDied","Data":"45ba5abface4d71436b7119fce2cd1a1aee3972c5909bfbc992e907cb2e2b6a1"} Jul 06 14:45:06.644523 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:06.644338 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45ba5abface4d71436b7119fce2cd1a1aee3972c5909bfbc992e907cb2e2b6a1" Jul 06 14:45:14.825832 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:14.825736 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:45:23.935007 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:23.934966 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:45:35.826422 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:35.826377 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:45:45.627057 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:45:45.627013 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:46:07.930511 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:46:07.930475 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:46:19.713627 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:46:19.713586 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:46:45.057435 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:46:45.057323 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:46:55.631633 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:46:55.631585 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:47:10.616833 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:47:10.616798 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:47:20.822943 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:47:20.822907 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:47:29.726232 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:47:29.726189 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:47:39.934619 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:47:39.934580 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:47:48.531218 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:47:48.531170 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:48:53.237020 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:48:53.236981 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:48:53.239608 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:48:53.239584 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:48:53.241319 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:48:53.241278 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:48:53.243982 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:48:53.243963 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:49:39.048856 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:49:39.048823 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:49:50.431116 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:49:50.431020 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:50:00.047117 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:50:00.047080 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:50:12.128291 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:50:12.128246 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:50:23.431647 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:50:23.431607 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:50:32.525131 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:50:32.525093 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:50:43.225607 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:50:43.225565 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:50:53.634093 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:50:53.634053 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:51:03.923511 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:51:03.923473 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:51:13.761251 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:51:13.761195 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:51:24.019199 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:51:24.019110 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:51:46.126693 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:51:46.126655 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:51:54.935062 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:51:54.935023 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:52:06.323632 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:52:06.323594 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:52:15.217978 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:52:15.217936 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:52:31.328672 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:52:31.328637 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:52:40.335516 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:52:40.335477 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:52:49.524246 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:52:49.524153 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:52:59.430779 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:52:59.430741 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:53:08.539440 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:53:08.539402 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:53:17.753847 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:53:17.753805 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:53:31.717865 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:53:31.717827 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:53:40.727637 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:53:40.727602 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:53:50.438804 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:53:50.438765 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:53:53.265332 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:53:53.265302 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:53:53.269274 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:53:53.269249 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:53:53.269663 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:53:53.269646 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:53:53.273434 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:53:53.273418 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:54:02.118056 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:54:02.118017 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:54:14.725766 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:54:14.725727 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:54:23.332575 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:54:23.332537 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:54:34.924024 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:54:34.923979 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:54:46.230537 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:54:46.230494 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:54:56.916695 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:54:56.916658 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:55:09.112290 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:55:09.112250 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:55:17.822084 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:55:17.822049 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:55:29.643592 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:55:29.643550 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:55:47.738927 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:55:47.738889 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:55:59.424268 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:55:59.424224 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:56:07.223059 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:56:07.223024 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:56:17.442887 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:56:17.442847 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:57:45.045272 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:57:45.045180 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:57:52.653271 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:57:52.653230 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:58:00.069668 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:00.069632 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:58:09.821995 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:09.821947 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:58:24.032228 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:24.032173 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:58:53.292990 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:53.292963 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:58:53.297383 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:53.297360 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:58:53.297999 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:53.297980 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 14:58:53.304975 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:53.304958 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 14:58:54.927453 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:54.927420 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:58:59.638364 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.638330 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw"] Jul 06 14:58:59.638800 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.638679 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adf65a14-adcd-42a7-8ea8-ba0ed4977a02" containerName="cleanup" Jul 06 14:58:59.638800 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.638692 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="adf65a14-adcd-42a7-8ea8-ba0ed4977a02" containerName="cleanup" Jul 06 14:58:59.638800 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.638746 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="adf65a14-adcd-42a7-8ea8-ba0ed4977a02" containerName="cleanup" Jul 06 14:58:59.641745 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.641728 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.646595 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.646562 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5737c8f3\"/\"default-dockercfg-qgx66\"" Jul 06 14:58:59.646732 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.646615 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5737c8f3\"/\"openshift-service-ca.crt\"" Jul 06 14:58:59.647631 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.647609 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5737c8f3\"/\"kube-root-ca.crt\"" Jul 06 14:58:59.647883 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.647868 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5737c8f3\"/\"e2e-lifecycle-model-5737c8f3-kserve-self-signed-certs\"" Jul 06 14:58:59.653069 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.653048 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw"] Jul 06 14:58:59.742528 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.742489 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-dshm\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.742528 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.742528 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-tmp-dir\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.742729 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.742546 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-model-cache\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.742729 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.742620 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgksf\" (UniqueName: \"kubernetes.io/projected/c9fe425e-f94b-4557-97b9-72928b4e7d63-kube-api-access-xgksf\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.742729 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.742689 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-home\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.742729 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.742723 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9fe425e-f94b-4557-97b9-72928b4e7d63-tls-certs\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.843781 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.843745 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-home\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.843781 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.843786 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9fe425e-f94b-4557-97b9-72928b4e7d63-tls-certs\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.843991 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.843818 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-dshm\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.843991 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.843839 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-tmp-dir\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.843991 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.843864 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-model-cache\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.843991 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.843899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgksf\" (UniqueName: \"kubernetes.io/projected/c9fe425e-f94b-4557-97b9-72928b4e7d63-kube-api-access-xgksf\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.844232 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.844177 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-home\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.844351 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.844333 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-tmp-dir\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.844448 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.844428 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-model-cache\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.846062 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.846042 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-dshm\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.846351 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.846335 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9fe425e-f94b-4557-97b9-72928b4e7d63-tls-certs\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.854138 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.854114 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgksf\" (UniqueName: \"kubernetes.io/projected/c9fe425e-f94b-4557-97b9-72928b4e7d63-kube-api-access-xgksf\") pod \"e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:58:59.950945 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:58:59.950862 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:59:00.086840 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:00.086811 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw"] Jul 06 14:59:00.089201 ip-10-0-137-102 kubenswrapper[2569]: W0706 14:59:00.089172 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9fe425e_f94b_4557_97b9_72928b4e7d63.slice/crio-84ff455740631a3a3a28bf0386f72bcb19e5df49cd4cee5cc79e85d74ce48574 WatchSource:0}: Error finding container 84ff455740631a3a3a28bf0386f72bcb19e5df49cd4cee5cc79e85d74ce48574: Status 404 returned error can't find the container with id 84ff455740631a3a3a28bf0386f72bcb19e5df49cd4cee5cc79e85d74ce48574 Jul 06 14:59:00.564280 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:00.564235 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" event={"ID":"c9fe425e-f94b-4557-97b9-72928b4e7d63","Type":"ContainerStarted","Data":"41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba"} Jul 06 14:59:00.564561 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:00.564301 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" event={"ID":"c9fe425e-f94b-4557-97b9-72928b4e7d63","Type":"ContainerStarted","Data":"84ff455740631a3a3a28bf0386f72bcb19e5df49cd4cee5cc79e85d74ce48574"} Jul 06 14:59:00.585639 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:00.585586 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" podStartSLOduration=1.585572408 podStartE2EDuration="1.585572408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:58:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:59:00.583790997 +0000 UTC m=+1808.224760928" watchObservedRunningTime="2026-07-06 14:59:00.585572408 +0000 UTC m=+1808.226542468" Jul 06 14:59:05.214379 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:05.214339 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:59:09.951711 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:09.951671 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:59:09.952109 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:09.951727 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:59:09.964237 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:09.964198 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:59:10.622903 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:10.622866 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:59:19.631062 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:19.631023 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:59:30.232423 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:30.232388 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 14:59:33.145951 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:33.145918 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw"] Jul 06 14:59:48.166922 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.166855 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" podUID="c9fe425e-f94b-4557-97b9-72928b4e7d63" containerName="main" containerID="cri-o://41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba" gracePeriod=45 Jul 06 14:59:48.413404 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.413379 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:59:48.456138 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.456041 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-home\") pod \"c9fe425e-f94b-4557-97b9-72928b4e7d63\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " Jul 06 14:59:48.456138 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.456111 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-model-cache\") pod \"c9fe425e-f94b-4557-97b9-72928b4e7d63\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " Jul 06 14:59:48.456382 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.456145 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-dshm\") pod \"c9fe425e-f94b-4557-97b9-72928b4e7d63\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " Jul 06 14:59:48.456382 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.456180 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9fe425e-f94b-4557-97b9-72928b4e7d63-tls-certs\") pod \"c9fe425e-f94b-4557-97b9-72928b4e7d63\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " Jul 06 14:59:48.456382 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.456271 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgksf\" (UniqueName: \"kubernetes.io/projected/c9fe425e-f94b-4557-97b9-72928b4e7d63-kube-api-access-xgksf\") pod \"c9fe425e-f94b-4557-97b9-72928b4e7d63\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " Jul 06 14:59:48.456382 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.456301 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-tmp-dir\") pod \"c9fe425e-f94b-4557-97b9-72928b4e7d63\" (UID: \"c9fe425e-f94b-4557-97b9-72928b4e7d63\") " Jul 06 14:59:48.456565 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.456405 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-model-cache" (OuterVolumeSpecName: "model-cache") pod "c9fe425e-f94b-4557-97b9-72928b4e7d63" (UID: "c9fe425e-f94b-4557-97b9-72928b4e7d63"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:59:48.456565 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.456395 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-home" (OuterVolumeSpecName: "home") pod "c9fe425e-f94b-4557-97b9-72928b4e7d63" (UID: "c9fe425e-f94b-4557-97b9-72928b4e7d63"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:59:48.456658 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.456567 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-home\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:59:48.456658 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.456587 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-model-cache\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:59:48.456752 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.456663 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c9fe425e-f94b-4557-97b9-72928b4e7d63" (UID: "c9fe425e-f94b-4557-97b9-72928b4e7d63"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:59:48.458657 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.458628 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9fe425e-f94b-4557-97b9-72928b4e7d63-kube-api-access-xgksf" (OuterVolumeSpecName: "kube-api-access-xgksf") pod "c9fe425e-f94b-4557-97b9-72928b4e7d63" (UID: "c9fe425e-f94b-4557-97b9-72928b4e7d63"). InnerVolumeSpecName "kube-api-access-xgksf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:59:48.458657 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.458646 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9fe425e-f94b-4557-97b9-72928b4e7d63-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c9fe425e-f94b-4557-97b9-72928b4e7d63" (UID: "c9fe425e-f94b-4557-97b9-72928b4e7d63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:59:48.458798 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.458689 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-dshm" (OuterVolumeSpecName: "dshm") pod "c9fe425e-f94b-4557-97b9-72928b4e7d63" (UID: "c9fe425e-f94b-4557-97b9-72928b4e7d63"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:59:48.558002 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.557963 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgksf\" (UniqueName: \"kubernetes.io/projected/c9fe425e-f94b-4557-97b9-72928b4e7d63-kube-api-access-xgksf\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:59:48.558002 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.557995 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-tmp-dir\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:59:48.558002 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.558007 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9fe425e-f94b-4557-97b9-72928b4e7d63-dshm\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:59:48.558002 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.558015 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9fe425e-f94b-4557-97b9-72928b4e7d63-tls-certs\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 14:59:48.787093 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.787057 2569 generic.go:356] "Generic (PLEG): container finished" podID="c9fe425e-f94b-4557-97b9-72928b4e7d63" containerID="41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba" exitCode=0 Jul 06 14:59:48.787277 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.787108 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" Jul 06 14:59:48.787277 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.787139 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" event={"ID":"c9fe425e-f94b-4557-97b9-72928b4e7d63","Type":"ContainerDied","Data":"41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba"} Jul 06 14:59:48.787277 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.787180 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw" event={"ID":"c9fe425e-f94b-4557-97b9-72928b4e7d63","Type":"ContainerDied","Data":"84ff455740631a3a3a28bf0386f72bcb19e5df49cd4cee5cc79e85d74ce48574"} Jul 06 14:59:48.787277 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.787224 2569 scope.go:117] "RemoveContainer" containerID="41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba" Jul 06 14:59:48.796263 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.796240 2569 scope.go:117] "RemoveContainer" containerID="41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba" Jul 06 14:59:48.796518 ip-10-0-137-102 kubenswrapper[2569]: E0706 14:59:48.796484 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba\": container with ID starting with 41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba not found: ID does not exist" containerID="41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba" Jul 06 14:59:48.796568 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.796525 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba"} err="failed to get container status \"41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba\": rpc error: code = NotFound desc = could not find container \"41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba\": container with ID starting with 41e832b80e21c9b45b087043de0781359dcb6bf08b13662bd05451229c7514ba not found: ID does not exist" Jul 06 14:59:48.811242 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.811216 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw"] Jul 06 14:59:48.816224 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:48.816190 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-5737c8f3/e2e-lifecycle-model-5737c8f3-kserve-74444869b7-lgjdw"] Jul 06 14:59:49.237716 ip-10-0-137-102 kubenswrapper[2569]: I0706 14:59:49.237636 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9fe425e-f94b-4557-97b9-72928b4e7d63" path="/var/lib/kubelet/pods/c9fe425e-f94b-4557-97b9-72928b4e7d63/volumes" Jul 06 15:00:00.162914 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.162876 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722500-9vz9k"] Jul 06 15:00:00.163323 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.163242 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9fe425e-f94b-4557-97b9-72928b4e7d63" containerName="main" Jul 06 15:00:00.163323 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.163255 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9fe425e-f94b-4557-97b9-72928b4e7d63" containerName="main" Jul 06 15:00:00.163323 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.163313 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9fe425e-f94b-4557-97b9-72928b4e7d63" containerName="main" Jul 06 15:00:00.166187 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.166168 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" Jul 06 15:00:00.169988 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.169968 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2zlpp\"" Jul 06 15:00:00.186398 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.186374 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722500-9vz9k"] Jul 06 15:00:00.254864 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.254813 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crkps\" (UniqueName: \"kubernetes.io/projected/1b5338fe-97e2-400d-97a5-9bc218c3eea2-kube-api-access-crkps\") pod \"maas-api-key-cleanup-29722500-9vz9k\" (UID: \"1b5338fe-97e2-400d-97a5-9bc218c3eea2\") " pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" Jul 06 15:00:00.355767 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.355721 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crkps\" (UniqueName: \"kubernetes.io/projected/1b5338fe-97e2-400d-97a5-9bc218c3eea2-kube-api-access-crkps\") pod \"maas-api-key-cleanup-29722500-9vz9k\" (UID: \"1b5338fe-97e2-400d-97a5-9bc218c3eea2\") " pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" Jul 06 15:00:00.371718 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.371689 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crkps\" (UniqueName: \"kubernetes.io/projected/1b5338fe-97e2-400d-97a5-9bc218c3eea2-kube-api-access-crkps\") pod \"maas-api-key-cleanup-29722500-9vz9k\" (UID: \"1b5338fe-97e2-400d-97a5-9bc218c3eea2\") " pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" Jul 06 15:00:00.476797 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.476687 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" Jul 06 15:00:00.612797 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.612769 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722500-9vz9k"] Jul 06 15:00:00.615657 ip-10-0-137-102 kubenswrapper[2569]: W0706 15:00:00.615627 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5338fe_97e2_400d_97a5_9bc218c3eea2.slice/crio-2222d64084476335875f6ec2c2ee4e745d57767d33eeb2234810214cb737489f WatchSource:0}: Error finding container 2222d64084476335875f6ec2c2ee4e745d57767d33eeb2234810214cb737489f: Status 404 returned error can't find the container with id 2222d64084476335875f6ec2c2ee4e745d57767d33eeb2234810214cb737489f Jul 06 15:00:00.842539 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.842505 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" event={"ID":"1b5338fe-97e2-400d-97a5-9bc218c3eea2","Type":"ContainerStarted","Data":"f20ab1e2254d517b6011a983c27d45240429b34cbb2ae39d25dbbd7c74e8f02e"} Jul 06 15:00:00.842711 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.842559 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" event={"ID":"1b5338fe-97e2-400d-97a5-9bc218c3eea2","Type":"ContainerStarted","Data":"2222d64084476335875f6ec2c2ee4e745d57767d33eeb2234810214cb737489f"} Jul 06 15:00:00.863334 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:00.863275 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" podStartSLOduration=0.863257522 podStartE2EDuration="863.257522ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:00:00.862821544 +0000 UTC m=+1868.503791476" watchObservedRunningTime="2026-07-06 15:00:00.863257522 +0000 UTC m=+1868.504227455" Jul 06 15:00:01.848177 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:01.848143 2569 generic.go:356] "Generic (PLEG): container finished" podID="1b5338fe-97e2-400d-97a5-9bc218c3eea2" containerID="f20ab1e2254d517b6011a983c27d45240429b34cbb2ae39d25dbbd7c74e8f02e" exitCode=0 Jul 06 15:00:01.848177 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:01.848181 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" event={"ID":"1b5338fe-97e2-400d-97a5-9bc218c3eea2","Type":"ContainerDied","Data":"f20ab1e2254d517b6011a983c27d45240429b34cbb2ae39d25dbbd7c74e8f02e"} Jul 06 15:00:02.987754 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:02.987730 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" Jul 06 15:00:03.078815 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:03.078776 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crkps\" (UniqueName: \"kubernetes.io/projected/1b5338fe-97e2-400d-97a5-9bc218c3eea2-kube-api-access-crkps\") pod \"1b5338fe-97e2-400d-97a5-9bc218c3eea2\" (UID: \"1b5338fe-97e2-400d-97a5-9bc218c3eea2\") " Jul 06 15:00:03.080936 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:03.080915 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b5338fe-97e2-400d-97a5-9bc218c3eea2-kube-api-access-crkps" (OuterVolumeSpecName: "kube-api-access-crkps") pod "1b5338fe-97e2-400d-97a5-9bc218c3eea2" (UID: "1b5338fe-97e2-400d-97a5-9bc218c3eea2"). InnerVolumeSpecName "kube-api-access-crkps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:00:03.180266 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:03.180143 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crkps\" (UniqueName: \"kubernetes.io/projected/1b5338fe-97e2-400d-97a5-9bc218c3eea2-kube-api-access-crkps\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:00:03.858389 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:03.858357 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" Jul 06 15:00:03.858389 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:03.858365 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722500-9vz9k" event={"ID":"1b5338fe-97e2-400d-97a5-9bc218c3eea2","Type":"ContainerDied","Data":"2222d64084476335875f6ec2c2ee4e745d57767d33eeb2234810214cb737489f"} Jul 06 15:00:03.858588 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:03.858410 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2222d64084476335875f6ec2c2ee4e745d57767d33eeb2234810214cb737489f" Jul 06 15:00:11.921890 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:11.921855 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:00:35.351099 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:35.351014 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:00:47.587168 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:00:47.587130 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:01:41.825674 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:41.825636 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:01:42.218303 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.218228 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj"] Jul 06 15:01:42.218667 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.218650 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b5338fe-97e2-400d-97a5-9bc218c3eea2" containerName="cleanup" Jul 06 15:01:42.218667 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.218669 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b5338fe-97e2-400d-97a5-9bc218c3eea2" containerName="cleanup" Jul 06 15:01:42.218781 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.218743 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b5338fe-97e2-400d-97a5-9bc218c3eea2" containerName="cleanup" Jul 06 15:01:42.221851 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.221833 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:42.224826 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.224806 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-69ca3d-serving-cert\"" Jul 06 15:01:42.224947 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.224823 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2zlpp\"" Jul 06 15:01:42.232044 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.232018 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj"] Jul 06 15:01:42.401731 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.401693 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-openshift-service-ca\") pod \"maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:42.401935 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.401832 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nq9p\" (UniqueName: \"kubernetes.io/projected/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-kube-api-access-4nq9p\") pod \"maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:42.401935 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.401873 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-maas-api-tls\") pod \"maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:42.502855 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.502772 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-openshift-service-ca\") pod \"maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:42.502855 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.502813 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nq9p\" (UniqueName: \"kubernetes.io/projected/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-kube-api-access-4nq9p\") pod \"maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:42.502855 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.502833 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-maas-api-tls\") pod \"maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:42.503509 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.503487 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-openshift-service-ca\") pod \"maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:42.505153 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.505126 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-maas-api-tls\") pod \"maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:42.513006 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.512981 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nq9p\" (UniqueName: \"kubernetes.io/projected/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-kube-api-access-4nq9p\") pod \"maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:42.533966 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.533945 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:42.671313 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.671286 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj"] Jul 06 15:01:42.673609 ip-10-0-137-102 kubenswrapper[2569]: W0706 15:01:42.673576 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod470f3c35_9640_4eb8_99cb_2e2bcbb2e051.slice/crio-9961c2c55af66a02678daa577961c4ecbb45c8b79ea91094ff9866b7d09edb46 WatchSource:0}: Error finding container 9961c2c55af66a02678daa577961c4ecbb45c8b79ea91094ff9866b7d09edb46: Status 404 returned error can't find the container with id 9961c2c55af66a02678daa577961c4ecbb45c8b79ea91094ff9866b7d09edb46 Jul 06 15:01:42.674896 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:42.674877 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 15:01:43.334592 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:43.334559 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" event={"ID":"470f3c35-9640-4eb8-99cb-2e2bcbb2e051","Type":"ContainerStarted","Data":"9961c2c55af66a02678daa577961c4ecbb45c8b79ea91094ff9866b7d09edb46"} Jul 06 15:01:45.346583 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:45.346489 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" event={"ID":"470f3c35-9640-4eb8-99cb-2e2bcbb2e051","Type":"ContainerStarted","Data":"37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b"} Jul 06 15:01:45.346951 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:45.346577 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:51.356042 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:51.356016 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:01:51.376052 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:51.376005 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" podStartSLOduration=6.9996230310000005 podStartE2EDuration="9.375992648s" totalImagesPullingTime="2.376369617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:01:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:01:45.373754972 +0000 UTC m=+1973.014724904" watchObservedRunningTime="2026-07-06 15:01:51.375992648 +0000 UTC m=+1979.016962579" Jul 06 15:01:53.844978 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.844915 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg"] Jul 06 15:01:53.850459 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.850437 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.854402 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.854381 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-419f91\"/\"default-dockercfg-zzdbv\"" Jul 06 15:01:53.854536 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.854381 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-419f91\"/\"test-model-419f91-kserve-self-signed-certs\"" Jul 06 15:01:53.854536 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.854381 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-419f91\"/\"openshift-service-ca.crt\"" Jul 06 15:01:53.854536 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.854457 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-419f91\"/\"kube-root-ca.crt\"" Jul 06 15:01:53.858178 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.858152 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg"] Jul 06 15:01:53.893310 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.893277 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-model-cache\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.893433 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.893325 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-home\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.893433 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.893351 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-tmp-dir\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.893433 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.893404 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4nbd\" (UniqueName: \"kubernetes.io/projected/6c01174f-46b1-471e-aeab-e59444e0755b-kube-api-access-s4nbd\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.893544 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.893447 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-dshm\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.893544 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.893483 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c01174f-46b1-471e-aeab-e59444e0755b-tls-certs\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.994280 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.994240 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4nbd\" (UniqueName: \"kubernetes.io/projected/6c01174f-46b1-471e-aeab-e59444e0755b-kube-api-access-s4nbd\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.994446 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.994294 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-dshm\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.994446 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.994329 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c01174f-46b1-471e-aeab-e59444e0755b-tls-certs\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.994446 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.994406 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-model-cache\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.994619 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.994455 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-home\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.994619 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.994487 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-tmp-dir\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.994809 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.994783 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-model-cache\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.994924 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.994823 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-home\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.994924 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.994882 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-tmp-dir\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.996693 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.996673 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-dshm\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:53.996883 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:53.996866 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c01174f-46b1-471e-aeab-e59444e0755b-tls-certs\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:54.004332 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:54.004310 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4nbd\" (UniqueName: \"kubernetes.io/projected/6c01174f-46b1-471e-aeab-e59444e0755b-kube-api-access-s4nbd\") pod \"test-model-419f91-kserve-86bb558845-mzzlg\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:54.160648 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:54.160571 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:01:54.300143 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:54.300114 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg"] Jul 06 15:01:54.302033 ip-10-0-137-102 kubenswrapper[2569]: W0706 15:01:54.302007 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c01174f_46b1_471e_aeab_e59444e0755b.slice/crio-13c7cc8f76ece652c319fcbc43897fcee1bb619d30c093e7c5833cca087205b9 WatchSource:0}: Error finding container 13c7cc8f76ece652c319fcbc43897fcee1bb619d30c093e7c5833cca087205b9: Status 404 returned error can't find the container with id 13c7cc8f76ece652c319fcbc43897fcee1bb619d30c093e7c5833cca087205b9 Jul 06 15:01:54.387955 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:54.387923 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" event={"ID":"6c01174f-46b1-471e-aeab-e59444e0755b","Type":"ContainerStarted","Data":"183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400"} Jul 06 15:01:54.388090 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:54.387965 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" event={"ID":"6c01174f-46b1-471e-aeab-e59444e0755b","Type":"ContainerStarted","Data":"13c7cc8f76ece652c319fcbc43897fcee1bb619d30c093e7c5833cca087205b9"} Jul 06 15:01:54.408693 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:54.408635 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" podStartSLOduration=1.408619234 podStartE2EDuration="1.408619234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:01:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:01:54.407128578 +0000 UTC m=+1982.048098532" watchObservedRunningTime="2026-07-06 15:01:54.408619234 +0000 UTC m=+1982.049589178" Jul 06 15:01:55.211717 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:01:55.211683 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:02:04.160992 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:04.160958 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:02:04.161396 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:04.161013 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:02:04.173556 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:04.173531 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:02:04.448351 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:04.448269 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:02:10.637558 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:10.637520 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:02:19.151477 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.151437 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs"] Jul 06 15:02:19.153903 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.153884 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.159320 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.159296 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-69ca3d\"/\"openshift-service-ca.crt\"" Jul 06 15:02:19.159621 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.159600 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-69ca3d\"/\"default-dockercfg-mtqz5\"" Jul 06 15:02:19.160996 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.160740 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-69ca3d\"/\"test-model-69ca3d-kserve-self-signed-certs\"" Jul 06 15:02:19.160996 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.160761 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-69ca3d\"/\"kube-root-ca.crt\"" Jul 06 15:02:19.177480 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.177453 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs"] Jul 06 15:02:19.213423 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.213384 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8q4p\" (UniqueName: \"kubernetes.io/projected/d3810ae1-654d-4b1d-b123-e7452d4225b4-kube-api-access-k8q4p\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.213423 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.213425 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-home\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.213636 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.213490 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3810ae1-654d-4b1d-b123-e7452d4225b4-tls-certs\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.213636 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.213506 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-model-cache\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.213636 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.213563 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-dshm\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.213636 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.213608 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-tmp-dir\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.314412 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.314379 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-dshm\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.314412 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.314414 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-tmp-dir\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.314659 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.314461 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8q4p\" (UniqueName: \"kubernetes.io/projected/d3810ae1-654d-4b1d-b123-e7452d4225b4-kube-api-access-k8q4p\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.314659 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.314491 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-home\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.314659 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.314555 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3810ae1-654d-4b1d-b123-e7452d4225b4-tls-certs\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.314659 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.314578 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-model-cache\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.314880 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.314835 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-tmp-dir\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.314980 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.314958 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-home\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.315040 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.314964 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-model-cache\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.316706 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.316683 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-dshm\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.316987 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.316970 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3810ae1-654d-4b1d-b123-e7452d4225b4-tls-certs\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.326174 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.326150 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8q4p\" (UniqueName: \"kubernetes.io/projected/d3810ae1-654d-4b1d-b123-e7452d4225b4-kube-api-access-k8q4p\") pod \"test-model-69ca3d-kserve-6df8cfd489-ts4bs\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.463523 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.463433 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:19.599029 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:19.598987 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs"] Jul 06 15:02:19.600976 ip-10-0-137-102 kubenswrapper[2569]: W0706 15:02:19.600945 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3810ae1_654d_4b1d_b123_e7452d4225b4.slice/crio-9539844c974d9e671d16ed8131aace2968dfe23a38cb27b45b43bff24392770e WatchSource:0}: Error finding container 9539844c974d9e671d16ed8131aace2968dfe23a38cb27b45b43bff24392770e: Status 404 returned error can't find the container with id 9539844c974d9e671d16ed8131aace2968dfe23a38cb27b45b43bff24392770e Jul 06 15:02:20.511519 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:20.511482 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" event={"ID":"d3810ae1-654d-4b1d-b123-e7452d4225b4","Type":"ContainerStarted","Data":"467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca"} Jul 06 15:02:20.511519 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:20.511526 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" event={"ID":"d3810ae1-654d-4b1d-b123-e7452d4225b4","Type":"ContainerStarted","Data":"9539844c974d9e671d16ed8131aace2968dfe23a38cb27b45b43bff24392770e"} Jul 06 15:02:20.534457 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:20.534395 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" podStartSLOduration=1.5343767659999998 podStartE2EDuration="1.534376766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:02:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:02:20.533719491 +0000 UTC m=+2008.174689425" watchObservedRunningTime="2026-07-06 15:02:20.534376766 +0000 UTC m=+2008.175346698" Jul 06 15:02:24.744478 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:24.744438 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:02:29.463973 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:29.463924 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:29.463973 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:29.463989 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:29.476504 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:29.476475 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:29.574522 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:29.574487 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:37.248760 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:37.248727 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:02:40.321637 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:40.321604 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg"] Jul 06 15:02:41.319533 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:41.319503 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs"] Jul 06 15:02:52.732318 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:52.732283 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:02:55.342053 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.342007 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" podUID="6c01174f-46b1-471e-aeab-e59444e0755b" containerName="main" containerID="cri-o://183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400" gracePeriod=45 Jul 06 15:02:55.602275 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.602192 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:02:55.624257 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.624188 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-tmp-dir\") pod \"6c01174f-46b1-471e-aeab-e59444e0755b\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " Jul 06 15:02:55.624409 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.624275 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-model-cache\") pod \"6c01174f-46b1-471e-aeab-e59444e0755b\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " Jul 06 15:02:55.624409 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.624307 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-dshm\") pod \"6c01174f-46b1-471e-aeab-e59444e0755b\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " Jul 06 15:02:55.624409 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.624374 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4nbd\" (UniqueName: \"kubernetes.io/projected/6c01174f-46b1-471e-aeab-e59444e0755b-kube-api-access-s4nbd\") pod \"6c01174f-46b1-471e-aeab-e59444e0755b\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " Jul 06 15:02:55.624571 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.624430 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-home\") pod \"6c01174f-46b1-471e-aeab-e59444e0755b\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " Jul 06 15:02:55.624571 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.624456 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c01174f-46b1-471e-aeab-e59444e0755b-tls-certs\") pod \"6c01174f-46b1-471e-aeab-e59444e0755b\" (UID: \"6c01174f-46b1-471e-aeab-e59444e0755b\") " Jul 06 15:02:55.624571 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.624463 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6c01174f-46b1-471e-aeab-e59444e0755b" (UID: "6c01174f-46b1-471e-aeab-e59444e0755b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:02:55.624571 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.624491 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-model-cache" (OuterVolumeSpecName: "model-cache") pod "6c01174f-46b1-471e-aeab-e59444e0755b" (UID: "6c01174f-46b1-471e-aeab-e59444e0755b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:02:55.624771 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.624683 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-home" (OuterVolumeSpecName: "home") pod "6c01174f-46b1-471e-aeab-e59444e0755b" (UID: "6c01174f-46b1-471e-aeab-e59444e0755b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:02:55.624771 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.624694 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-tmp-dir\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:55.624771 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.624714 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-model-cache\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:55.626988 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.626958 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-dshm" (OuterVolumeSpecName: "dshm") pod "6c01174f-46b1-471e-aeab-e59444e0755b" (UID: "6c01174f-46b1-471e-aeab-e59444e0755b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:02:55.627104 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.627072 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c01174f-46b1-471e-aeab-e59444e0755b-kube-api-access-s4nbd" (OuterVolumeSpecName: "kube-api-access-s4nbd") pod "6c01174f-46b1-471e-aeab-e59444e0755b" (UID: "6c01174f-46b1-471e-aeab-e59444e0755b"). InnerVolumeSpecName "kube-api-access-s4nbd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:02:55.627182 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.627162 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c01174f-46b1-471e-aeab-e59444e0755b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6c01174f-46b1-471e-aeab-e59444e0755b" (UID: "6c01174f-46b1-471e-aeab-e59444e0755b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:02:55.701011 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.700974 2569 generic.go:356] "Generic (PLEG): container finished" podID="6c01174f-46b1-471e-aeab-e59444e0755b" containerID="183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400" exitCode=0 Jul 06 15:02:55.701184 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.701021 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" Jul 06 15:02:55.701184 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.701062 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" event={"ID":"6c01174f-46b1-471e-aeab-e59444e0755b","Type":"ContainerDied","Data":"183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400"} Jul 06 15:02:55.701184 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.701106 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg" event={"ID":"6c01174f-46b1-471e-aeab-e59444e0755b","Type":"ContainerDied","Data":"13c7cc8f76ece652c319fcbc43897fcee1bb619d30c093e7c5833cca087205b9"} Jul 06 15:02:55.701184 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.701123 2569 scope.go:117] "RemoveContainer" containerID="183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400" Jul 06 15:02:55.709870 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.709854 2569 scope.go:117] "RemoveContainer" containerID="183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400" Jul 06 15:02:55.710125 ip-10-0-137-102 kubenswrapper[2569]: E0706 15:02:55.710103 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400\": container with ID starting with 183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400 not found: ID does not exist" containerID="183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400" Jul 06 15:02:55.710168 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.710134 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400"} err="failed to get container status \"183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400\": rpc error: code = NotFound desc = could not find container \"183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400\": container with ID starting with 183a9cf76cde9242dffbf82ee1f50e7a0f3720fc8479d4171809730509e46400 not found: ID does not exist" Jul 06 15:02:55.725899 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.725876 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg"] Jul 06 15:02:55.726130 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.726112 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-dshm\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:55.726173 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.726134 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4nbd\" (UniqueName: \"kubernetes.io/projected/6c01174f-46b1-471e-aeab-e59444e0755b-kube-api-access-s4nbd\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:55.726173 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.726144 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c01174f-46b1-471e-aeab-e59444e0755b-home\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:55.726173 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.726153 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c01174f-46b1-471e-aeab-e59444e0755b-tls-certs\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:55.732376 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:55.732357 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-419f91/test-model-419f91-kserve-86bb558845-mzzlg"] Jul 06 15:02:56.339999 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.339940 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" podUID="d3810ae1-654d-4b1d-b123-e7452d4225b4" containerName="main" containerID="cri-o://467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca" gracePeriod=45 Jul 06 15:02:56.593793 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.593738 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:56.635114 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.635083 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-home\") pod \"d3810ae1-654d-4b1d-b123-e7452d4225b4\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " Jul 06 15:02:56.635292 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.635149 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8q4p\" (UniqueName: \"kubernetes.io/projected/d3810ae1-654d-4b1d-b123-e7452d4225b4-kube-api-access-k8q4p\") pod \"d3810ae1-654d-4b1d-b123-e7452d4225b4\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " Jul 06 15:02:56.635292 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.635171 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-model-cache\") pod \"d3810ae1-654d-4b1d-b123-e7452d4225b4\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " Jul 06 15:02:56.635292 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.635188 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-dshm\") pod \"d3810ae1-654d-4b1d-b123-e7452d4225b4\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " Jul 06 15:02:56.635292 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.635280 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3810ae1-654d-4b1d-b123-e7452d4225b4-tls-certs\") pod \"d3810ae1-654d-4b1d-b123-e7452d4225b4\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " Jul 06 15:02:56.635516 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.635311 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-tmp-dir\") pod \"d3810ae1-654d-4b1d-b123-e7452d4225b4\" (UID: \"d3810ae1-654d-4b1d-b123-e7452d4225b4\") " Jul 06 15:02:56.635516 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.635451 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-model-cache" (OuterVolumeSpecName: "model-cache") pod "d3810ae1-654d-4b1d-b123-e7452d4225b4" (UID: "d3810ae1-654d-4b1d-b123-e7452d4225b4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:02:56.635516 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.635463 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-home" (OuterVolumeSpecName: "home") pod "d3810ae1-654d-4b1d-b123-e7452d4225b4" (UID: "d3810ae1-654d-4b1d-b123-e7452d4225b4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:02:56.635682 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.635661 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d3810ae1-654d-4b1d-b123-e7452d4225b4" (UID: "d3810ae1-654d-4b1d-b123-e7452d4225b4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:02:56.635739 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.635684 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-model-cache\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:56.635739 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.635700 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-home\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:56.637442 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.637415 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-dshm" (OuterVolumeSpecName: "dshm") pod "d3810ae1-654d-4b1d-b123-e7452d4225b4" (UID: "d3810ae1-654d-4b1d-b123-e7452d4225b4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:02:56.637548 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.637522 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3810ae1-654d-4b1d-b123-e7452d4225b4-kube-api-access-k8q4p" (OuterVolumeSpecName: "kube-api-access-k8q4p") pod "d3810ae1-654d-4b1d-b123-e7452d4225b4" (UID: "d3810ae1-654d-4b1d-b123-e7452d4225b4"). InnerVolumeSpecName "kube-api-access-k8q4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:02:56.637674 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.637657 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3810ae1-654d-4b1d-b123-e7452d4225b4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d3810ae1-654d-4b1d-b123-e7452d4225b4" (UID: "d3810ae1-654d-4b1d-b123-e7452d4225b4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:02:56.708840 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.708805 2569 generic.go:356] "Generic (PLEG): container finished" podID="d3810ae1-654d-4b1d-b123-e7452d4225b4" containerID="467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca" exitCode=0 Jul 06 15:02:56.709006 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.708854 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" Jul 06 15:02:56.709006 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.708857 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" event={"ID":"d3810ae1-654d-4b1d-b123-e7452d4225b4","Type":"ContainerDied","Data":"467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca"} Jul 06 15:02:56.709006 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.708895 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs" event={"ID":"d3810ae1-654d-4b1d-b123-e7452d4225b4","Type":"ContainerDied","Data":"9539844c974d9e671d16ed8131aace2968dfe23a38cb27b45b43bff24392770e"} Jul 06 15:02:56.709006 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.708915 2569 scope.go:117] "RemoveContainer" containerID="467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca" Jul 06 15:02:56.717887 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.717871 2569 scope.go:117] "RemoveContainer" containerID="467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca" Jul 06 15:02:56.718138 ip-10-0-137-102 kubenswrapper[2569]: E0706 15:02:56.718123 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca\": container with ID starting with 467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca not found: ID does not exist" containerID="467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca" Jul 06 15:02:56.718183 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.718146 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca"} err="failed to get container status \"467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca\": rpc error: code = NotFound desc = could not find container \"467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca\": container with ID starting with 467c7abb09e81eae3afba2ec9f0eb5e148364200d1dc0fc442520c37d4b986ca not found: ID does not exist" Jul 06 15:02:56.734262 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.734235 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs"] Jul 06 15:02:56.736651 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.736633 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8q4p\" (UniqueName: \"kubernetes.io/projected/d3810ae1-654d-4b1d-b123-e7452d4225b4-kube-api-access-k8q4p\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:56.736704 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.736653 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-dshm\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:56.736704 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.736663 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3810ae1-654d-4b1d-b123-e7452d4225b4-tls-certs\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:56.736704 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.736672 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3810ae1-654d-4b1d-b123-e7452d4225b4-tmp-dir\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:02:56.740176 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:56.740156 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-69ca3d/test-model-69ca3d-kserve-6df8cfd489-ts4bs"] Jul 06 15:02:57.236813 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:57.236771 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c01174f-46b1-471e-aeab-e59444e0755b" path="/var/lib/kubelet/pods/6c01174f-46b1-471e-aeab-e59444e0755b/volumes" Jul 06 15:02:57.237282 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:02:57.237263 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3810ae1-654d-4b1d-b123-e7452d4225b4" path="/var/lib/kubelet/pods/d3810ae1-654d-4b1d-b123-e7452d4225b4/volumes" Jul 06 15:03:26.355583 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.355488 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj"] Jul 06 15:03:26.356125 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.355733 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" podUID="470f3c35-9640-4eb8-99cb-2e2bcbb2e051" containerName="maas-api" containerID="cri-o://37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b" gracePeriod=30 Jul 06 15:03:26.609666 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.609600 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:03:26.799529 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.799496 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nq9p\" (UniqueName: \"kubernetes.io/projected/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-kube-api-access-4nq9p\") pod \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " Jul 06 15:03:26.799724 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.799544 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-maas-api-tls\") pod \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " Jul 06 15:03:26.799724 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.799589 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-openshift-service-ca\") pod \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\" (UID: \"470f3c35-9640-4eb8-99cb-2e2bcbb2e051\") " Jul 06 15:03:26.799998 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.799973 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "470f3c35-9640-4eb8-99cb-2e2bcbb2e051" (UID: "470f3c35-9640-4eb8-99cb-2e2bcbb2e051"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:03:26.801634 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.801605 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "470f3c35-9640-4eb8-99cb-2e2bcbb2e051" (UID: "470f3c35-9640-4eb8-99cb-2e2bcbb2e051"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:03:26.801736 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.801647 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-kube-api-access-4nq9p" (OuterVolumeSpecName: "kube-api-access-4nq9p") pod "470f3c35-9640-4eb8-99cb-2e2bcbb2e051" (UID: "470f3c35-9640-4eb8-99cb-2e2bcbb2e051"). InnerVolumeSpecName "kube-api-access-4nq9p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:03:26.853021 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.852991 2569 generic.go:356] "Generic (PLEG): container finished" podID="470f3c35-9640-4eb8-99cb-2e2bcbb2e051" containerID="37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b" exitCode=0 Jul 06 15:03:26.853147 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.853039 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" Jul 06 15:03:26.853147 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.853031 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" event={"ID":"470f3c35-9640-4eb8-99cb-2e2bcbb2e051","Type":"ContainerDied","Data":"37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b"} Jul 06 15:03:26.853303 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.853185 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj" event={"ID":"470f3c35-9640-4eb8-99cb-2e2bcbb2e051","Type":"ContainerDied","Data":"9961c2c55af66a02678daa577961c4ecbb45c8b79ea91094ff9866b7d09edb46"} Jul 06 15:03:26.853303 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.853240 2569 scope.go:117] "RemoveContainer" containerID="37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b" Jul 06 15:03:26.862010 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.861992 2569 scope.go:117] "RemoveContainer" containerID="37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b" Jul 06 15:03:26.862294 ip-10-0-137-102 kubenswrapper[2569]: E0706 15:03:26.862268 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b\": container with ID starting with 37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b not found: ID does not exist" containerID="37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b" Jul 06 15:03:26.862348 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.862308 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b"} err="failed to get container status \"37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b\": rpc error: code = NotFound desc = could not find container \"37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b\": container with ID starting with 37e88e81f12098958716cebf920cedfb7fe86e93d61f0cad605d12a59d28677b not found: ID does not exist" Jul 06 15:03:26.878303 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.878281 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj"] Jul 06 15:03:26.881459 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.881438 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-69ca3d-5d78989f74-7nmmj"] Jul 06 15:03:26.900867 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.900849 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nq9p\" (UniqueName: \"kubernetes.io/projected/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-kube-api-access-4nq9p\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:03:26.900939 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.900870 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-maas-api-tls\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:03:26.900939 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:26.900880 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/470f3c35-9640-4eb8-99cb-2e2bcbb2e051-openshift-service-ca\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:03:27.238035 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:27.237958 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="470f3c35-9640-4eb8-99cb-2e2bcbb2e051" path="/var/lib/kubelet/pods/470f3c35-9640-4eb8-99cb-2e2bcbb2e051/volumes" Jul 06 15:03:28.919421 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:28.919384 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:03:36.890731 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.890699 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng"] Jul 06 15:03:36.891100 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.891024 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="470f3c35-9640-4eb8-99cb-2e2bcbb2e051" containerName="maas-api" Jul 06 15:03:36.891100 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.891034 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="470f3c35-9640-4eb8-99cb-2e2bcbb2e051" containerName="maas-api" Jul 06 15:03:36.891100 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.891043 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3810ae1-654d-4b1d-b123-e7452d4225b4" containerName="main" Jul 06 15:03:36.891100 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.891050 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3810ae1-654d-4b1d-b123-e7452d4225b4" containerName="main" Jul 06 15:03:36.891100 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.891057 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c01174f-46b1-471e-aeab-e59444e0755b" containerName="main" Jul 06 15:03:36.891100 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.891063 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c01174f-46b1-471e-aeab-e59444e0755b" containerName="main" Jul 06 15:03:36.891301 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.891129 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3810ae1-654d-4b1d-b123-e7452d4225b4" containerName="main" Jul 06 15:03:36.891301 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.891136 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="470f3c35-9640-4eb8-99cb-2e2bcbb2e051" containerName="maas-api" Jul 06 15:03:36.891301 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.891146 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c01174f-46b1-471e-aeab-e59444e0755b" containerName="main" Jul 06 15:03:36.894319 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.894296 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:36.901036 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.901008 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-b5a326-serving-cert\"" Jul 06 15:03:36.901237 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.901187 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2zlpp\"" Jul 06 15:03:36.911065 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.911033 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng"] Jul 06 15:03:36.984057 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.984022 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2307b3-0567-4661-8a29-471310d868a2-maas-api-tls\") pod \"maas-api-e2e-api-a-b5a326-b57697465-csrng\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:36.984057 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.984062 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2307b3-0567-4661-8a29-471310d868a2-openshift-service-ca\") pod \"maas-api-e2e-api-a-b5a326-b57697465-csrng\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:36.984301 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:36.984157 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8kpv\" (UniqueName: \"kubernetes.io/projected/1b2307b3-0567-4661-8a29-471310d868a2-kube-api-access-d8kpv\") pod \"maas-api-e2e-api-a-b5a326-b57697465-csrng\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:37.085098 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.085062 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2307b3-0567-4661-8a29-471310d868a2-maas-api-tls\") pod \"maas-api-e2e-api-a-b5a326-b57697465-csrng\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:37.085098 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.085097 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2307b3-0567-4661-8a29-471310d868a2-openshift-service-ca\") pod \"maas-api-e2e-api-a-b5a326-b57697465-csrng\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:37.085389 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.085139 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8kpv\" (UniqueName: \"kubernetes.io/projected/1b2307b3-0567-4661-8a29-471310d868a2-kube-api-access-d8kpv\") pod \"maas-api-e2e-api-a-b5a326-b57697465-csrng\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:37.085858 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.085835 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2307b3-0567-4661-8a29-471310d868a2-openshift-service-ca\") pod \"maas-api-e2e-api-a-b5a326-b57697465-csrng\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:37.087566 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.087540 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2307b3-0567-4661-8a29-471310d868a2-maas-api-tls\") pod \"maas-api-e2e-api-a-b5a326-b57697465-csrng\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:37.097521 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.097500 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8kpv\" (UniqueName: \"kubernetes.io/projected/1b2307b3-0567-4661-8a29-471310d868a2-kube-api-access-d8kpv\") pod \"maas-api-e2e-api-a-b5a326-b57697465-csrng\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:37.204945 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.204862 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:37.351110 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.351083 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng"] Jul 06 15:03:37.353065 ip-10-0-137-102 kubenswrapper[2569]: W0706 15:03:37.353041 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b2307b3_0567_4661_8a29_471310d868a2.slice/crio-34c0a302d9a8f54cda378ee519987d898e2d7f4813a2d1708ed7ddde4a1408e8 WatchSource:0}: Error finding container 34c0a302d9a8f54cda378ee519987d898e2d7f4813a2d1708ed7ddde4a1408e8: Status 404 returned error can't find the container with id 34c0a302d9a8f54cda378ee519987d898e2d7f4813a2d1708ed7ddde4a1408e8 Jul 06 15:03:37.910133 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.910101 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" event={"ID":"1b2307b3-0567-4661-8a29-471310d868a2","Type":"ContainerStarted","Data":"530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2"} Jul 06 15:03:37.910502 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.910155 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" event={"ID":"1b2307b3-0567-4661-8a29-471310d868a2","Type":"ContainerStarted","Data":"34c0a302d9a8f54cda378ee519987d898e2d7f4813a2d1708ed7ddde4a1408e8"} Jul 06 15:03:37.910502 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.910194 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:37.937426 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:37.937376 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" podStartSLOduration=1.46303115 podStartE2EDuration="1.937360321s" totalImagesPullingTime="474.329171ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:03:37.934931117 +0000 UTC m=+2085.575901048" watchObservedRunningTime="2026-07-06 15:03:37.937360321 +0000 UTC m=+2085.578330290" Jul 06 15:03:42.424663 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:42.424630 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:03:43.920255 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:43.920224 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:45.158280 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.158240 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp"] Jul 06 15:03:45.162105 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.162079 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:45.164705 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.164682 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-1da456-serving-cert\"" Jul 06 15:03:45.174299 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.174269 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp"] Jul 06 15:03:45.250670 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.250636 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19a04bab-1378-4bb5-9fa9-c4356531a526-maas-api-tls\") pod \"maas-api-e2e-api-b-1da456-545d745fcd-7wvhp\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:45.250844 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.250680 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8td92\" (UniqueName: \"kubernetes.io/projected/19a04bab-1378-4bb5-9fa9-c4356531a526-kube-api-access-8td92\") pod \"maas-api-e2e-api-b-1da456-545d745fcd-7wvhp\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:45.250844 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.250733 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a04bab-1378-4bb5-9fa9-c4356531a526-openshift-service-ca\") pod \"maas-api-e2e-api-b-1da456-545d745fcd-7wvhp\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:45.351720 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.351682 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8td92\" (UniqueName: \"kubernetes.io/projected/19a04bab-1378-4bb5-9fa9-c4356531a526-kube-api-access-8td92\") pod \"maas-api-e2e-api-b-1da456-545d745fcd-7wvhp\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:45.351887 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.351730 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a04bab-1378-4bb5-9fa9-c4356531a526-openshift-service-ca\") pod \"maas-api-e2e-api-b-1da456-545d745fcd-7wvhp\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:45.351887 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.351785 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19a04bab-1378-4bb5-9fa9-c4356531a526-maas-api-tls\") pod \"maas-api-e2e-api-b-1da456-545d745fcd-7wvhp\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:45.352430 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.352410 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a04bab-1378-4bb5-9fa9-c4356531a526-openshift-service-ca\") pod \"maas-api-e2e-api-b-1da456-545d745fcd-7wvhp\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:45.354100 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.354081 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19a04bab-1378-4bb5-9fa9-c4356531a526-maas-api-tls\") pod \"maas-api-e2e-api-b-1da456-545d745fcd-7wvhp\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:45.361676 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.361650 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8td92\" (UniqueName: \"kubernetes.io/projected/19a04bab-1378-4bb5-9fa9-c4356531a526-kube-api-access-8td92\") pod \"maas-api-e2e-api-b-1da456-545d745fcd-7wvhp\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:45.474819 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.474728 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:45.613789 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.613760 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp"] Jul 06 15:03:45.615361 ip-10-0-137-102 kubenswrapper[2569]: W0706 15:03:45.615333 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a04bab_1378_4bb5_9fa9_c4356531a526.slice/crio-61d612463e993c03b53140c5fc460d8f3bcb88da0f7b5e7247eaf8d12a6c39df WatchSource:0}: Error finding container 61d612463e993c03b53140c5fc460d8f3bcb88da0f7b5e7247eaf8d12a6c39df: Status 404 returned error can't find the container with id 61d612463e993c03b53140c5fc460d8f3bcb88da0f7b5e7247eaf8d12a6c39df Jul 06 15:03:45.953999 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:45.953964 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" event={"ID":"19a04bab-1378-4bb5-9fa9-c4356531a526","Type":"ContainerStarted","Data":"61d612463e993c03b53140c5fc460d8f3bcb88da0f7b5e7247eaf8d12a6c39df"} Jul 06 15:03:46.960728 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:46.960683 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" event={"ID":"19a04bab-1378-4bb5-9fa9-c4356531a526","Type":"ContainerStarted","Data":"af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37"} Jul 06 15:03:46.961117 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:46.960748 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:46.981090 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:46.981036 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" podStartSLOduration=1.58493661 podStartE2EDuration="1.981021892s" totalImagesPullingTime="396.085282ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:03:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:03:46.979768314 +0000 UTC m=+2094.620738272" watchObservedRunningTime="2026-07-06 15:03:46.981021892 +0000 UTC m=+2094.621991823" Jul 06 15:03:52.970226 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:52.970173 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:03:53.321934 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:53.321908 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 15:03:53.326565 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:53.326541 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 15:03:53.330785 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:53.330764 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 15:03:53.334613 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:53.334596 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 15:03:58.135668 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:58.135631 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:03:58.685991 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:58.685954 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng"] Jul 06 15:03:58.686280 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:58.686200 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" podUID="1b2307b3-0567-4661-8a29-471310d868a2" containerName="maas-api" containerID="cri-o://530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2" gracePeriod=30 Jul 06 15:03:58.931372 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:58.931340 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:59.020550 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.020516 2569 generic.go:356] "Generic (PLEG): container finished" podID="1b2307b3-0567-4661-8a29-471310d868a2" containerID="530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2" exitCode=0 Jul 06 15:03:59.020706 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.020606 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" event={"ID":"1b2307b3-0567-4661-8a29-471310d868a2","Type":"ContainerDied","Data":"530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2"} Jul 06 15:03:59.020706 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.020664 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" event={"ID":"1b2307b3-0567-4661-8a29-471310d868a2","Type":"ContainerDied","Data":"34c0a302d9a8f54cda378ee519987d898e2d7f4813a2d1708ed7ddde4a1408e8"} Jul 06 15:03:59.020706 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.020688 2569 scope.go:117] "RemoveContainer" containerID="530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2" Jul 06 15:03:59.020706 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.020584 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" Jul 06 15:03:59.029105 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.029090 2569 scope.go:117] "RemoveContainer" containerID="530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2" Jul 06 15:03:59.029375 ip-10-0-137-102 kubenswrapper[2569]: E0706 15:03:59.029353 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2\": container with ID starting with 530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2 not found: ID does not exist" containerID="530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2" Jul 06 15:03:59.029472 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.029383 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2"} err="failed to get container status \"530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2\": rpc error: code = NotFound desc = could not find container \"530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2\": container with ID starting with 530bc2f4b3c11de5dc7a8ca403c37b5130d6ac41034949948219b58ba2ae7dd2 not found: ID does not exist" Jul 06 15:03:59.063665 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.063642 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2307b3-0567-4661-8a29-471310d868a2-maas-api-tls\") pod \"1b2307b3-0567-4661-8a29-471310d868a2\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " Jul 06 15:03:59.063777 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.063708 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8kpv\" (UniqueName: \"kubernetes.io/projected/1b2307b3-0567-4661-8a29-471310d868a2-kube-api-access-d8kpv\") pod \"1b2307b3-0567-4661-8a29-471310d868a2\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " Jul 06 15:03:59.063777 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.063736 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2307b3-0567-4661-8a29-471310d868a2-openshift-service-ca\") pod \"1b2307b3-0567-4661-8a29-471310d868a2\" (UID: \"1b2307b3-0567-4661-8a29-471310d868a2\") " Jul 06 15:03:59.064131 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.064107 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b2307b3-0567-4661-8a29-471310d868a2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1b2307b3-0567-4661-8a29-471310d868a2" (UID: "1b2307b3-0567-4661-8a29-471310d868a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:03:59.065785 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.065752 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b2307b3-0567-4661-8a29-471310d868a2-kube-api-access-d8kpv" (OuterVolumeSpecName: "kube-api-access-d8kpv") pod "1b2307b3-0567-4661-8a29-471310d868a2" (UID: "1b2307b3-0567-4661-8a29-471310d868a2"). InnerVolumeSpecName "kube-api-access-d8kpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:03:59.065868 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.065761 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b2307b3-0567-4661-8a29-471310d868a2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1b2307b3-0567-4661-8a29-471310d868a2" (UID: "1b2307b3-0567-4661-8a29-471310d868a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:03:59.165450 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.165408 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8kpv\" (UniqueName: \"kubernetes.io/projected/1b2307b3-0567-4661-8a29-471310d868a2-kube-api-access-d8kpv\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:03:59.165450 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.165442 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2307b3-0567-4661-8a29-471310d868a2-openshift-service-ca\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:03:59.165450 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.165456 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2307b3-0567-4661-8a29-471310d868a2-maas-api-tls\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:03:59.351751 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.351674 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng"] Jul 06 15:03:59.355121 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:03:59.355095 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng"] Jul 06 15:04:01.236795 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:01.236760 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b2307b3-0567-4661-8a29-471310d868a2" path="/var/lib/kubelet/pods/1b2307b3-0567-4661-8a29-471310d868a2/volumes" Jul 06 15:04:01.916182 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:01.916139 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-api-a-b5a326-b57697465-csrng" podUID="1b2307b3-0567-4661-8a29-471310d868a2" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.39:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 06 15:04:06.250669 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.250625 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp"] Jul 06 15:04:06.251141 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.250905 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" podUID="19a04bab-1378-4bb5-9fa9-c4356531a526" containerName="maas-api" containerID="cri-o://af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37" gracePeriod=30 Jul 06 15:04:06.502639 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.502579 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:04:06.624586 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.624553 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8td92\" (UniqueName: \"kubernetes.io/projected/19a04bab-1378-4bb5-9fa9-c4356531a526-kube-api-access-8td92\") pod \"19a04bab-1378-4bb5-9fa9-c4356531a526\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " Jul 06 15:04:06.624762 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.624615 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19a04bab-1378-4bb5-9fa9-c4356531a526-maas-api-tls\") pod \"19a04bab-1378-4bb5-9fa9-c4356531a526\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " Jul 06 15:04:06.624762 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.624646 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a04bab-1378-4bb5-9fa9-c4356531a526-openshift-service-ca\") pod \"19a04bab-1378-4bb5-9fa9-c4356531a526\" (UID: \"19a04bab-1378-4bb5-9fa9-c4356531a526\") " Jul 06 15:04:06.625087 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.625064 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19a04bab-1378-4bb5-9fa9-c4356531a526-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "19a04bab-1378-4bb5-9fa9-c4356531a526" (UID: "19a04bab-1378-4bb5-9fa9-c4356531a526"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:04:06.626695 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.626669 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19a04bab-1378-4bb5-9fa9-c4356531a526-kube-api-access-8td92" (OuterVolumeSpecName: "kube-api-access-8td92") pod "19a04bab-1378-4bb5-9fa9-c4356531a526" (UID: "19a04bab-1378-4bb5-9fa9-c4356531a526"). InnerVolumeSpecName "kube-api-access-8td92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:04:06.626773 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.626677 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19a04bab-1378-4bb5-9fa9-c4356531a526-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "19a04bab-1378-4bb5-9fa9-c4356531a526" (UID: "19a04bab-1378-4bb5-9fa9-c4356531a526"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:04:06.725956 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.725919 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a04bab-1378-4bb5-9fa9-c4356531a526-openshift-service-ca\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:04:06.725956 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.725953 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8td92\" (UniqueName: \"kubernetes.io/projected/19a04bab-1378-4bb5-9fa9-c4356531a526-kube-api-access-8td92\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:04:06.725956 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:06.725962 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19a04bab-1378-4bb5-9fa9-c4356531a526-maas-api-tls\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:04:07.072613 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:07.072576 2569 generic.go:356] "Generic (PLEG): container finished" podID="19a04bab-1378-4bb5-9fa9-c4356531a526" containerID="af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37" exitCode=0 Jul 06 15:04:07.072794 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:07.072620 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" Jul 06 15:04:07.072794 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:07.072663 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" event={"ID":"19a04bab-1378-4bb5-9fa9-c4356531a526","Type":"ContainerDied","Data":"af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37"} Jul 06 15:04:07.072794 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:07.072714 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp" event={"ID":"19a04bab-1378-4bb5-9fa9-c4356531a526","Type":"ContainerDied","Data":"61d612463e993c03b53140c5fc460d8f3bcb88da0f7b5e7247eaf8d12a6c39df"} Jul 06 15:04:07.072794 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:07.072735 2569 scope.go:117] "RemoveContainer" containerID="af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37" Jul 06 15:04:07.082182 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:07.082163 2569 scope.go:117] "RemoveContainer" containerID="af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37" Jul 06 15:04:07.082448 ip-10-0-137-102 kubenswrapper[2569]: E0706 15:04:07.082426 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37\": container with ID starting with af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37 not found: ID does not exist" containerID="af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37" Jul 06 15:04:07.082514 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:07.082459 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37"} err="failed to get container status \"af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37\": rpc error: code = NotFound desc = could not find container \"af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37\": container with ID starting with af858e6c7a7a7bf28d8de8e54a0125099371b2414c09d8ae7fdff2f506bd9d37 not found: ID does not exist" Jul 06 15:04:07.100835 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:07.100810 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp"] Jul 06 15:04:07.107397 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:07.107373 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-1da456-545d745fcd-7wvhp"] Jul 06 15:04:07.236521 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:07.236489 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19a04bab-1378-4bb5-9fa9-c4356531a526" path="/var/lib/kubelet/pods/19a04bab-1378-4bb5-9fa9-c4356531a526/volumes" Jul 06 15:04:09.837540 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:09.837507 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:04:16.223088 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.223044 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d"] Jul 06 15:04:16.223590 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.223426 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b2307b3-0567-4661-8a29-471310d868a2" containerName="maas-api" Jul 06 15:04:16.223590 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.223439 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b2307b3-0567-4661-8a29-471310d868a2" containerName="maas-api" Jul 06 15:04:16.223590 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.223449 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a04bab-1378-4bb5-9fa9-c4356531a526" containerName="maas-api" Jul 06 15:04:16.223590 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.223455 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a04bab-1378-4bb5-9fa9-c4356531a526" containerName="maas-api" Jul 06 15:04:16.223590 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.223527 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a04bab-1378-4bb5-9fa9-c4356531a526" containerName="maas-api" Jul 06 15:04:16.223590 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.223538 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b2307b3-0567-4661-8a29-471310d868a2" containerName="maas-api" Jul 06 15:04:16.227997 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.227980 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:16.230833 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.230809 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2zlpp\"" Jul 06 15:04:16.230936 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.230839 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-296ba3-serving-cert\"" Jul 06 15:04:16.239079 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.239056 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d"] Jul 06 15:04:16.308552 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.308519 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:16.308552 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.308552 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-openshift-service-ca\") pod \"maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:16.308836 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.308612 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkpjk\" (UniqueName: \"kubernetes.io/projected/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-kube-api-access-hkpjk\") pod \"maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:16.409883 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.409843 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkpjk\" (UniqueName: \"kubernetes.io/projected/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-kube-api-access-hkpjk\") pod \"maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:16.410046 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.409945 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:16.410046 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.409976 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-openshift-service-ca\") pod \"maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:16.410124 ip-10-0-137-102 kubenswrapper[2569]: E0706 15:04:16.410084 2569 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-296ba3-serving-cert: secret "maas-api-e2e-shared-a-296ba3-serving-cert" not found Jul 06 15:04:16.410173 ip-10-0-137-102 kubenswrapper[2569]: E0706 15:04:16.410162 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-maas-api-tls podName:68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea nodeName:}" failed. No retries permitted until 2026-07-06 15:04:16.91014572 +0000 UTC m=+2124.551115631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-maas-api-tls") pod "maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" (UID: "68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea") : secret "maas-api-e2e-shared-a-296ba3-serving-cert" not found Jul 06 15:04:16.410695 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.410678 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-openshift-service-ca\") pod \"maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:16.424324 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.424297 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkpjk\" (UniqueName: \"kubernetes.io/projected/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-kube-api-access-hkpjk\") pod \"maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:16.913868 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.913837 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:16.916252 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:16.916222 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:17.138678 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:17.138644 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:17.304088 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:17.304058 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d"] Jul 06 15:04:17.305310 ip-10-0-137-102 kubenswrapper[2569]: W0706 15:04:17.305281 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d5dbdc_f12e_4242_b1e3_ab4381f1f1ea.slice/crio-47568ac3deedf161e4d20405f3831473bf73d22706436f7bf5a721e57fc68c7e WatchSource:0}: Error finding container 47568ac3deedf161e4d20405f3831473bf73d22706436f7bf5a721e57fc68c7e: Status 404 returned error can't find the container with id 47568ac3deedf161e4d20405f3831473bf73d22706436f7bf5a721e57fc68c7e Jul 06 15:04:18.127475 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:18.127436 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" event={"ID":"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea","Type":"ContainerStarted","Data":"9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8"} Jul 06 15:04:18.127858 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:18.127485 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" event={"ID":"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea","Type":"ContainerStarted","Data":"47568ac3deedf161e4d20405f3831473bf73d22706436f7bf5a721e57fc68c7e"} Jul 06 15:04:18.127858 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:18.127624 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:20.734343 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:20.734284 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" podStartSLOduration=4.414948882 podStartE2EDuration="4.734265765s" totalImagesPullingTime="319.316883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:04:18.169708358 +0000 UTC m=+2125.810678288" watchObservedRunningTime="2026-07-06 15:04:20.734265765 +0000 UTC m=+2128.375235696" Jul 06 15:04:20.734786 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:20.734552 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:04:24.137879 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.137843 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:04:24.199228 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.199177 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2"] Jul 06 15:04:24.202888 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.202866 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:24.208324 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.208307 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-c7737a-serving-cert\"" Jul 06 15:04:24.218940 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.218909 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2"] Jul 06 15:04:24.377967 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.377935 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:24.377967 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.377970 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r25ft\" (UniqueName: \"kubernetes.io/projected/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-kube-api-access-r25ft\") pod \"maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:24.378190 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.377994 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-maas-api-tls\") pod \"maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:24.478697 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.478611 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:24.478697 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.478648 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r25ft\" (UniqueName: \"kubernetes.io/projected/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-kube-api-access-r25ft\") pod \"maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:24.478697 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.478672 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-maas-api-tls\") pod \"maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:24.479309 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.479285 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:24.480993 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.480976 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-maas-api-tls\") pod \"maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:24.489382 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.489361 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r25ft\" (UniqueName: \"kubernetes.io/projected/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-kube-api-access-r25ft\") pod \"maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:24.514344 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.514320 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:24.655120 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:24.655089 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2"] Jul 06 15:04:24.658856 ip-10-0-137-102 kubenswrapper[2569]: W0706 15:04:24.658827 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c17f1d5_74d8_4b06_aac6_e2ac9603bc76.slice/crio-d88e80bfc80f78c1c23096546471518e15b2ace1d026b4fe4fde0f43756b03e3 WatchSource:0}: Error finding container d88e80bfc80f78c1c23096546471518e15b2ace1d026b4fe4fde0f43756b03e3: Status 404 returned error can't find the container with id d88e80bfc80f78c1c23096546471518e15b2ace1d026b4fe4fde0f43756b03e3 Jul 06 15:04:25.162950 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:25.162917 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" event={"ID":"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76","Type":"ContainerStarted","Data":"a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5"} Jul 06 15:04:25.163278 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:25.162968 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" event={"ID":"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76","Type":"ContainerStarted","Data":"d88e80bfc80f78c1c23096546471518e15b2ace1d026b4fe4fde0f43756b03e3"} Jul 06 15:04:25.163278 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:25.163065 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:25.183884 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:25.183832 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" podStartSLOduration=0.796230071 podStartE2EDuration="1.183814884s" totalImagesPullingTime="387.584813ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:04:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:04:25.182094805 +0000 UTC m=+2132.823064737" watchObservedRunningTime="2026-07-06 15:04:25.183814884 +0000 UTC m=+2132.824784815" Jul 06 15:04:31.172811 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:31.172779 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:04:35.515830 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:35.515793 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:04:45.627498 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.627413 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656"] Jul 06 15:04:45.630788 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.630767 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.633383 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.633358 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-296ba3\"/\"default-dockercfg-jgmdp\"" Jul 06 15:04:45.634472 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.634452 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-296ba3\"/\"openshift-service-ca.crt\"" Jul 06 15:04:45.634598 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.634454 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-296ba3\"/\"kube-root-ca.crt\"" Jul 06 15:04:45.634598 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.634487 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-296ba3\"/\"auth-test-model-296ba3-kserve-self-signed-certs\"" Jul 06 15:04:45.643341 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.643316 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656"] Jul 06 15:04:45.654447 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.654420 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d372843a-edab-4ad4-a477-4ec71bab96c9-tls-certs\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.654552 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.654460 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-model-cache\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.654552 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.654477 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-home\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.654644 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.654555 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-tmp-dir\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.654644 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.654607 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x55qv\" (UniqueName: \"kubernetes.io/projected/d372843a-edab-4ad4-a477-4ec71bab96c9-kube-api-access-x55qv\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.654720 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.654644 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-dshm\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.755947 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.755909 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x55qv\" (UniqueName: \"kubernetes.io/projected/d372843a-edab-4ad4-a477-4ec71bab96c9-kube-api-access-x55qv\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.755947 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.755948 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-dshm\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.756246 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.755985 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d372843a-edab-4ad4-a477-4ec71bab96c9-tls-certs\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.756246 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.756010 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-model-cache\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.756246 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.756028 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-home\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.756246 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.756051 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-tmp-dir\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.756480 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.756416 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-model-cache\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.756532 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.756475 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-tmp-dir\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.756532 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.756501 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-home\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.758255 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.758232 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-dshm\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.758616 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.758597 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d372843a-edab-4ad4-a477-4ec71bab96c9-tls-certs\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.770662 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.770632 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x55qv\" (UniqueName: \"kubernetes.io/projected/d372843a-edab-4ad4-a477-4ec71bab96c9-kube-api-access-x55qv\") pod \"auth-test-model-296ba3-kserve-5bd684669f-c8656\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:45.941200 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:45.941104 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:46.079705 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:46.079675 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656"] Jul 06 15:04:46.081502 ip-10-0-137-102 kubenswrapper[2569]: W0706 15:04:46.081476 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd372843a_edab_4ad4_a477_4ec71bab96c9.slice/crio-647c93a45e608193e0b07404af3181cd803708970236b8525dd0f35104676ce2 WatchSource:0}: Error finding container 647c93a45e608193e0b07404af3181cd803708970236b8525dd0f35104676ce2: Status 404 returned error can't find the container with id 647c93a45e608193e0b07404af3181cd803708970236b8525dd0f35104676ce2 Jul 06 15:04:46.270408 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:46.270375 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" event={"ID":"d372843a-edab-4ad4-a477-4ec71bab96c9","Type":"ContainerStarted","Data":"7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08"} Jul 06 15:04:46.270584 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:46.270421 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" event={"ID":"d372843a-edab-4ad4-a477-4ec71bab96c9","Type":"ContainerStarted","Data":"647c93a45e608193e0b07404af3181cd803708970236b8525dd0f35104676ce2"} Jul 06 15:04:46.289098 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:46.289044 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" podStartSLOduration=1.28902813 podStartE2EDuration="1.28902813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:04:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:04:46.287713225 +0000 UTC m=+2153.928683184" watchObservedRunningTime="2026-07-06 15:04:46.28902813 +0000 UTC m=+2153.929998061" Jul 06 15:04:49.323709 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:49.323677 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:04:55.941463 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:55.941423 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:55.941946 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:55.941477 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:55.954137 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:55.954105 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:04:56.331393 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:04:56.331364 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:05:09.118669 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:05:09.118636 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:05:25.614586 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:05:25.614551 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:05:43.313555 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:05:43.313514 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:05:59.515837 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:05:59.515799 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:06:14.540165 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:06:14.540126 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:06:31.837835 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:06:31.837792 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:06:49.512549 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:06:49.512511 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:07:03.088636 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:03.088600 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656"] Jul 06 15:07:06.828317 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:06.828259 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:07:18.108718 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.108661 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" podUID="d372843a-edab-4ad4-a477-4ec71bab96c9" containerName="main" containerID="cri-o://7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08" gracePeriod=45 Jul 06 15:07:18.356581 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.356554 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:07:18.386734 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.386658 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-dshm\") pod \"d372843a-edab-4ad4-a477-4ec71bab96c9\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " Jul 06 15:07:18.386734 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.386695 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x55qv\" (UniqueName: \"kubernetes.io/projected/d372843a-edab-4ad4-a477-4ec71bab96c9-kube-api-access-x55qv\") pod \"d372843a-edab-4ad4-a477-4ec71bab96c9\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " Jul 06 15:07:18.386734 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.386723 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-home\") pod \"d372843a-edab-4ad4-a477-4ec71bab96c9\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " Jul 06 15:07:18.387010 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.386747 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-model-cache\") pod \"d372843a-edab-4ad4-a477-4ec71bab96c9\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " Jul 06 15:07:18.387010 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.386765 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-tmp-dir\") pod \"d372843a-edab-4ad4-a477-4ec71bab96c9\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " Jul 06 15:07:18.387010 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.386822 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d372843a-edab-4ad4-a477-4ec71bab96c9-tls-certs\") pod \"d372843a-edab-4ad4-a477-4ec71bab96c9\" (UID: \"d372843a-edab-4ad4-a477-4ec71bab96c9\") " Jul 06 15:07:18.387181 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.387016 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-home" (OuterVolumeSpecName: "home") pod "d372843a-edab-4ad4-a477-4ec71bab96c9" (UID: "d372843a-edab-4ad4-a477-4ec71bab96c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:07:18.387181 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.387069 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d372843a-edab-4ad4-a477-4ec71bab96c9" (UID: "d372843a-edab-4ad4-a477-4ec71bab96c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:07:18.387181 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.387083 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "d372843a-edab-4ad4-a477-4ec71bab96c9" (UID: "d372843a-edab-4ad4-a477-4ec71bab96c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:07:18.389003 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.388972 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-dshm" (OuterVolumeSpecName: "dshm") pod "d372843a-edab-4ad4-a477-4ec71bab96c9" (UID: "d372843a-edab-4ad4-a477-4ec71bab96c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:07:18.389003 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.388974 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d372843a-edab-4ad4-a477-4ec71bab96c9-kube-api-access-x55qv" (OuterVolumeSpecName: "kube-api-access-x55qv") pod "d372843a-edab-4ad4-a477-4ec71bab96c9" (UID: "d372843a-edab-4ad4-a477-4ec71bab96c9"). InnerVolumeSpecName "kube-api-access-x55qv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:07:18.389153 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.389078 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d372843a-edab-4ad4-a477-4ec71bab96c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d372843a-edab-4ad4-a477-4ec71bab96c9" (UID: "d372843a-edab-4ad4-a477-4ec71bab96c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:07:18.488142 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.488105 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-dshm\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:07:18.488142 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.488135 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x55qv\" (UniqueName: \"kubernetes.io/projected/d372843a-edab-4ad4-a477-4ec71bab96c9-kube-api-access-x55qv\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:07:18.488387 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.488151 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-home\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:07:18.488387 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.488163 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-model-cache\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:07:18.488387 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.488174 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d372843a-edab-4ad4-a477-4ec71bab96c9-tmp-dir\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:07:18.488387 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:18.488184 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d372843a-edab-4ad4-a477-4ec71bab96c9-tls-certs\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:07:19.005019 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:19.004982 2569 generic.go:356] "Generic (PLEG): container finished" podID="d372843a-edab-4ad4-a477-4ec71bab96c9" containerID="7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08" exitCode=0 Jul 06 15:07:19.005188 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:19.005029 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" Jul 06 15:07:19.005188 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:19.005069 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" event={"ID":"d372843a-edab-4ad4-a477-4ec71bab96c9","Type":"ContainerDied","Data":"7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08"} Jul 06 15:07:19.005188 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:19.005111 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656" event={"ID":"d372843a-edab-4ad4-a477-4ec71bab96c9","Type":"ContainerDied","Data":"647c93a45e608193e0b07404af3181cd803708970236b8525dd0f35104676ce2"} Jul 06 15:07:19.005188 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:19.005129 2569 scope.go:117] "RemoveContainer" containerID="7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08" Jul 06 15:07:19.016307 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:19.016229 2569 scope.go:117] "RemoveContainer" containerID="7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08" Jul 06 15:07:19.016750 ip-10-0-137-102 kubenswrapper[2569]: E0706 15:07:19.016721 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08\": container with ID starting with 7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08 not found: ID does not exist" containerID="7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08" Jul 06 15:07:19.016866 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:19.016755 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08"} err="failed to get container status \"7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08\": rpc error: code = NotFound desc = could not find container \"7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08\": container with ID starting with 7cb3e6bfd9677caaabc981082b98a31fc0b816862420174735e446adc799be08 not found: ID does not exist" Jul 06 15:07:19.029399 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:19.029372 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656"] Jul 06 15:07:19.037264 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:19.037244 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/auth-test-model-296ba3-kserve-5bd684669f-c8656"] Jul 06 15:07:19.237358 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:19.237327 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d372843a-edab-4ad4-a477-4ec71bab96c9" path="/var/lib/kubelet/pods/d372843a-edab-4ad4-a477-4ec71bab96c9/volumes" Jul 06 15:07:24.434527 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:07:24.434491 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:08:02.631663 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:02.631623 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:08:17.371698 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:17.371663 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:08:35.562623 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:35.562584 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:08:50.605131 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.605085 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz"] Jul 06 15:08:50.605525 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.605473 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d372843a-edab-4ad4-a477-4ec71bab96c9" containerName="main" Jul 06 15:08:50.605525 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.605487 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d372843a-edab-4ad4-a477-4ec71bab96c9" containerName="main" Jul 06 15:08:50.605620 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.605571 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d372843a-edab-4ad4-a477-4ec71bab96c9" containerName="main" Jul 06 15:08:50.608552 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.608533 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.612462 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.612437 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-296ba3\"/\"rate-test-model-296ba3-kserve-self-signed-certs\"" Jul 06 15:08:50.612581 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.612448 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-296ba3\"/\"kube-root-ca.crt\"" Jul 06 15:08:50.612837 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.612820 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-296ba3\"/\"openshift-service-ca.crt\"" Jul 06 15:08:50.613026 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.613010 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-296ba3\"/\"default-dockercfg-jgmdp\"" Jul 06 15:08:50.622499 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.622476 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz"] Jul 06 15:08:50.676397 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.676365 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-home\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.676397 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.676395 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tmp-dir\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.676631 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.676420 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-dshm\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.676631 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.676502 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-model-cache\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.676631 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.676540 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc9hn\" (UniqueName: \"kubernetes.io/projected/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-kube-api-access-pc9hn\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.676631 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.676611 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tls-certs\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.777257 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.777198 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-dshm\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.777426 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.777281 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-model-cache\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.777426 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.777308 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc9hn\" (UniqueName: \"kubernetes.io/projected/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-kube-api-access-pc9hn\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.777426 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.777380 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tls-certs\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.777558 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.777433 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-home\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.777558 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.777456 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tmp-dir\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.777737 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.777716 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-model-cache\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.777811 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.777758 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tmp-dir\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.777868 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.777823 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-home\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.779533 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.779507 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-dshm\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.779839 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.779822 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tls-certs\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.789049 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.789028 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc9hn\" (UniqueName: \"kubernetes.io/projected/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-kube-api-access-pc9hn\") pod \"rate-test-model-296ba3-kserve-55ff95974c-fm6zz\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:50.917832 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:50.917749 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:08:51.057575 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:51.057545 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz"] Jul 06 15:08:51.060068 ip-10-0-137-102 kubenswrapper[2569]: W0706 15:08:51.060040 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98b4e3f8_cf5c_4a7b_ac6c_8305c92368b2.slice/crio-4fad16e6450db85225b5425c3b28d9e7e9412436ae6a246cc4419b25e0d797e9 WatchSource:0}: Error finding container 4fad16e6450db85225b5425c3b28d9e7e9412436ae6a246cc4419b25e0d797e9: Status 404 returned error can't find the container with id 4fad16e6450db85225b5425c3b28d9e7e9412436ae6a246cc4419b25e0d797e9 Jul 06 15:08:51.447733 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:51.447699 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" event={"ID":"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2","Type":"ContainerStarted","Data":"f1d9a40fd6a3fa824120566f00979fb694687c6fe41c23e490cc1c9a43cc2d88"} Jul 06 15:08:51.448129 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:51.447743 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" event={"ID":"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2","Type":"ContainerStarted","Data":"4fad16e6450db85225b5425c3b28d9e7e9412436ae6a246cc4419b25e0d797e9"} Jul 06 15:08:51.469693 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:51.469640 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" podStartSLOduration=1.46962491 podStartE2EDuration="1.46962491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:08:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:08:51.467662879 +0000 UTC m=+2399.108632811" watchObservedRunningTime="2026-07-06 15:08:51.46962491 +0000 UTC m=+2399.110594841" Jul 06 15:08:53.351881 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:53.351854 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 15:08:53.362437 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:53.362400 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 15:08:53.367749 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:53.367723 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 15:08:53.371689 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:53.371669 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q7ks8_b81dc945-3aca-47e3-aacc-476e6704f594/ovn-acl-logging/0.log" Jul 06 15:08:54.276508 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:08:54.276476 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:09:00.918585 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:09:00.918530 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:09:00.919067 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:09:00.918860 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:09:00.931257 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:09:00.931225 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:09:01.509794 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:09:01.509765 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:09:52.839795 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:09:52.839759 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:10:06.239992 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:06.239952 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:10:20.329901 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:20.329860 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:10:27.739365 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:27.739327 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz"] Jul 06 15:10:30.858886 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:30.858846 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d"] Jul 06 15:10:30.859397 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:30.859180 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" podUID="68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea" containerName="maas-api" containerID="cri-o://9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8" gracePeriod=30 Jul 06 15:10:31.105940 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.105913 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:10:31.161891 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.161823 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-maas-api-tls\") pod \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " Jul 06 15:10:31.161891 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.161866 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-openshift-service-ca\") pod \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " Jul 06 15:10:31.162073 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.161900 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkpjk\" (UniqueName: \"kubernetes.io/projected/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-kube-api-access-hkpjk\") pod \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\" (UID: \"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea\") " Jul 06 15:10:31.162232 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.162195 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea" (UID: "68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:10:31.163825 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.163804 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea" (UID: "68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:10:31.163925 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.163911 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-kube-api-access-hkpjk" (OuterVolumeSpecName: "kube-api-access-hkpjk") pod "68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea" (UID: "68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea"). InnerVolumeSpecName "kube-api-access-hkpjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:10:31.263155 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.263121 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-maas-api-tls\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:10:31.263155 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.263150 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-openshift-service-ca\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:10:31.263155 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.263161 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkpjk\" (UniqueName: \"kubernetes.io/projected/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea-kube-api-access-hkpjk\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:10:31.946759 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.946723 2569 generic.go:356] "Generic (PLEG): container finished" podID="68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea" containerID="9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8" exitCode=0 Jul 06 15:10:31.947157 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.946763 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" event={"ID":"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea","Type":"ContainerDied","Data":"9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8"} Jul 06 15:10:31.947157 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.946825 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" event={"ID":"68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea","Type":"ContainerDied","Data":"47568ac3deedf161e4d20405f3831473bf73d22706436f7bf5a721e57fc68c7e"} Jul 06 15:10:31.947157 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.946847 2569 scope.go:117] "RemoveContainer" containerID="9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8" Jul 06 15:10:31.947157 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.946774 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d" Jul 06 15:10:31.955596 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.955577 2569 scope.go:117] "RemoveContainer" containerID="9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8" Jul 06 15:10:31.955842 ip-10-0-137-102 kubenswrapper[2569]: E0706 15:10:31.955824 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8\": container with ID starting with 9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8 not found: ID does not exist" containerID="9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8" Jul 06 15:10:31.955910 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.955850 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8"} err="failed to get container status \"9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8\": rpc error: code = NotFound desc = could not find container \"9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8\": container with ID starting with 9a241bcaef49bd8840fc6e2ac850be74b4fd3c0529825c05e065d268b9cdcec8 not found: ID does not exist" Jul 06 15:10:31.968799 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.968774 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d"] Jul 06 15:10:31.973547 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:31.973524 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-296ba3-5b4b495d6b-ghp6d"] Jul 06 15:10:33.236954 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:33.236920 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea" path="/var/lib/kubelet/pods/68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea/volumes" Jul 06 15:10:33.828934 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:33.828893 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:10:42.760118 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:42.760081 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" podUID="98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" containerName="main" containerID="cri-o://f1d9a40fd6a3fa824120566f00979fb694687c6fe41c23e490cc1c9a43cc2d88" gracePeriod=45 Jul 06 15:10:43.001455 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.001425 2569 generic.go:356] "Generic (PLEG): container finished" podID="98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" containerID="f1d9a40fd6a3fa824120566f00979fb694687c6fe41c23e490cc1c9a43cc2d88" exitCode=0 Jul 06 15:10:43.001578 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.001496 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" event={"ID":"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2","Type":"ContainerDied","Data":"f1d9a40fd6a3fa824120566f00979fb694687c6fe41c23e490cc1c9a43cc2d88"} Jul 06 15:10:43.001578 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.001537 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" event={"ID":"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2","Type":"ContainerDied","Data":"4fad16e6450db85225b5425c3b28d9e7e9412436ae6a246cc4419b25e0d797e9"} Jul 06 15:10:43.001578 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.001549 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fad16e6450db85225b5425c3b28d9e7e9412436ae6a246cc4419b25e0d797e9" Jul 06 15:10:43.006602 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.006584 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:10:43.066149 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066118 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tmp-dir\") pod \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " Jul 06 15:10:43.066149 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066151 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-model-cache\") pod \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " Jul 06 15:10:43.066408 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066175 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc9hn\" (UniqueName: \"kubernetes.io/projected/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-kube-api-access-pc9hn\") pod \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " Jul 06 15:10:43.066408 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066249 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-home\") pod \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " Jul 06 15:10:43.066408 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066274 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tls-certs\") pod \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " Jul 06 15:10:43.066408 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066308 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-dshm\") pod \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\" (UID: \"98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2\") " Jul 06 15:10:43.066587 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066464 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-model-cache" (OuterVolumeSpecName: "model-cache") pod "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" (UID: "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:10:43.066587 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066487 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" (UID: "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:10:43.066587 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066501 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-home" (OuterVolumeSpecName: "home") pod "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" (UID: "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:10:43.066686 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066605 2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-home\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:10:43.066686 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066621 2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tmp-dir\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:10:43.066686 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.066631 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-model-cache\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:10:43.068378 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.068356 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-kube-api-access-pc9hn" (OuterVolumeSpecName: "kube-api-access-pc9hn") pod "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" (UID: "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2"). InnerVolumeSpecName "kube-api-access-pc9hn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:10:43.068471 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.068379 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" (UID: "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:10:43.068471 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.068423 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-dshm" (OuterVolumeSpecName: "dshm") pod "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" (UID: "98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 15:10:43.168060 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.168021 2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-dshm\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:10:43.168060 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.168054 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc9hn\" (UniqueName: \"kubernetes.io/projected/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-kube-api-access-pc9hn\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:10:43.168060 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:43.168068 2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2-tls-certs\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:10:44.006529 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:44.006495 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz" Jul 06 15:10:44.032393 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:44.032363 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz"] Jul 06 15:10:44.039717 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:44.039687 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-296ba3/rate-test-model-296ba3-kserve-55ff95974c-fm6zz"] Jul 06 15:10:45.236943 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:10:45.236862 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" path="/var/lib/kubelet/pods/98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2/volumes" Jul 06 15:11:16.664641 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:16.664599 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2"] Jul 06 15:11:16.665136 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:16.664847 2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" podUID="9c17f1d5-74d8-4b06-aac6-e2ac9603bc76" containerName="maas-api" containerID="cri-o://a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5" gracePeriod=30 Jul 06 15:11:16.919789 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:16.919725 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:11:17.063480 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.063434 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r25ft\" (UniqueName: \"kubernetes.io/projected/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-kube-api-access-r25ft\") pod \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " Jul 06 15:11:17.063480 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.063490 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-openshift-service-ca\") pod \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " Jul 06 15:11:17.063714 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.063555 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-maas-api-tls\") pod \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\" (UID: \"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76\") " Jul 06 15:11:17.063949 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.063916 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c17f1d5-74d8-4b06-aac6-e2ac9603bc76" (UID: "9c17f1d5-74d8-4b06-aac6-e2ac9603bc76"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:11:17.065546 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.065526 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c17f1d5-74d8-4b06-aac6-e2ac9603bc76" (UID: "9c17f1d5-74d8-4b06-aac6-e2ac9603bc76"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:11:17.065649 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.065630 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-kube-api-access-r25ft" (OuterVolumeSpecName: "kube-api-access-r25ft") pod "9c17f1d5-74d8-4b06-aac6-e2ac9603bc76" (UID: "9c17f1d5-74d8-4b06-aac6-e2ac9603bc76"). InnerVolumeSpecName "kube-api-access-r25ft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:11:17.161116 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.161086 2569 generic.go:356] "Generic (PLEG): container finished" podID="9c17f1d5-74d8-4b06-aac6-e2ac9603bc76" containerID="a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5" exitCode=0 Jul 06 15:11:17.161266 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.161130 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" Jul 06 15:11:17.161266 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.161152 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" event={"ID":"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76","Type":"ContainerDied","Data":"a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5"} Jul 06 15:11:17.161266 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.161194 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2" event={"ID":"9c17f1d5-74d8-4b06-aac6-e2ac9603bc76","Type":"ContainerDied","Data":"d88e80bfc80f78c1c23096546471518e15b2ace1d026b4fe4fde0f43756b03e3"} Jul 06 15:11:17.161266 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.161229 2569 scope.go:117] "RemoveContainer" containerID="a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5" Jul 06 15:11:17.164723 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.164701 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r25ft\" (UniqueName: \"kubernetes.io/projected/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-kube-api-access-r25ft\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:11:17.164841 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.164726 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-openshift-service-ca\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:11:17.164841 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.164739 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76-maas-api-tls\") on node \"ip-10-0-137-102.ec2.internal\" DevicePath \"\"" Jul 06 15:11:17.173637 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.173617 2569 scope.go:117] "RemoveContainer" containerID="a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5" Jul 06 15:11:17.173928 ip-10-0-137-102 kubenswrapper[2569]: E0706 15:11:17.173905 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5\": container with ID starting with a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5 not found: ID does not exist" containerID="a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5" Jul 06 15:11:17.174028 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.173939 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5"} err="failed to get container status \"a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5\": rpc error: code = NotFound desc = could not find container \"a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5\": container with ID starting with a642999db17e443470089219e58bef6559381f3fdb16649e212049ad617a50f5 not found: ID does not exist" Jul 06 15:11:17.207218 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.207183 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2"] Jul 06 15:11:17.215262 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.215238 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-c7737a-b5547d68d-kfsg2"] Jul 06 15:11:17.237189 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:17.237164 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c17f1d5-74d8-4b06-aac6-e2ac9603bc76" path="/var/lib/kubelet/pods/9c17f1d5-74d8-4b06-aac6-e2ac9603bc76/volumes" Jul 06 15:11:19.358666 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:19.358626 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-htz88"] Jul 06 15:11:31.520552 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:31.520523 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722485-f45qj_adf65a14-adcd-42a7-8ea8-ba0ed4977a02/cleanup/0.log" Jul 06 15:11:31.635763 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:31.635736 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722500-9vz9k_1b5338fe-97e2-400d-97a5-9bc218c3eea2/cleanup/0.log" Jul 06 15:11:32.295883 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:32.295853 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75545b5d88-4swgm_2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05/copy-manifests/0.log" Jul 06 15:11:32.306884 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:32.306839 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75545b5d88-4swgm_2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05/manager/0.log" Jul 06 15:11:34.713727 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:34.713701 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-htz88_e1e49d62-0ca4-4f2c-9698-f7a450b97bb6/limitador/0.log" Jul 06 15:11:35.382256 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:35.382228 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xzsp6_3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3/discovery/0.log" Jul 06 15:11:36.244549 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:36.244520 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fd947d8df-9l5bt_c56d1fb4-a9ab-41e7-b8ab-dd83500ec477/router/0.log" Jul 06 15:11:37.198430 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:37.198403 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-w9ssq_62fe3ea9-bb50-4159-a278-c834a118a64b/main/0.log" Jul 06 15:11:45.129231 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:45.129186 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dzg6l_2e25332d-2afd-48fd-98a6-c42e1a6f93b9/global-pull-secret-syncer/0.log" Jul 06 15:11:45.350352 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:45.350320 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-75l8n_ed76f900-7a58-48b0-8cf8-d53fea1737f0/konnectivity-agent/0.log" Jul 06 15:11:45.505329 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:45.505250 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-102.ec2.internal_61c1ee201a8907e80dca74652a24fb31/haproxy/0.log" Jul 06 15:11:50.577043 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:50.577011 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-htz88_e1e49d62-0ca4-4f2c-9698-f7a450b97bb6/limitador/0.log" Jul 06 15:11:52.866201 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:52.866169 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-vs5xn_fae66ff7-10e5-4713-a0a7-a2bfb0246aba/cluster-monitoring-operator/0.log" Jul 06 15:11:52.942582 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:52.942548 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-4rwmc_930f0621-c03a-4d4c-b3ad-22305faef6a1/kube-state-metrics/0.log" Jul 06 15:11:53.034875 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:53.034848 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-4rwmc_930f0621-c03a-4d4c-b3ad-22305faef6a1/kube-rbac-proxy-main/0.log" Jul 06 15:11:53.096374 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:53.096351 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-4rwmc_930f0621-c03a-4d4c-b3ad-22305faef6a1/kube-rbac-proxy-self/0.log" Jul 06 15:11:53.202786 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:53.202712 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-csswz_9238a25b-bdde-4d14-9051-5ec571e146c5/monitoring-plugin/0.log" Jul 06 15:11:53.437250 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:53.437220 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7j2gj_7fb42739-96de-40b9-9ee4-1345292bc983/node-exporter/0.log" Jul 06 15:11:53.493630 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:53.493607 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7j2gj_7fb42739-96de-40b9-9ee4-1345292bc983/kube-rbac-proxy/0.log" Jul 06 15:11:53.540456 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:53.540434 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7j2gj_7fb42739-96de-40b9-9ee4-1345292bc983/init-textfile/0.log" Jul 06 15:11:54.141444 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.141411 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-x7h49_defedf5a-88aa-41a5-aba4-86e42864fe18/prometheus-operator/0.log" Jul 06 15:11:54.194633 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.194606 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-x7h49_defedf5a-88aa-41a5-aba4-86e42864fe18/kube-rbac-proxy/0.log" Jul 06 15:11:54.964635 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.964595 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn"] Jul 06 15:11:54.965003 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.964985 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c17f1d5-74d8-4b06-aac6-e2ac9603bc76" containerName="maas-api" Jul 06 15:11:54.965094 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.965006 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c17f1d5-74d8-4b06-aac6-e2ac9603bc76" containerName="maas-api" Jul 06 15:11:54.965094 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.965018 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea" containerName="maas-api" Jul 06 15:11:54.965094 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.965026 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea" containerName="maas-api" Jul 06 15:11:54.965094 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.965045 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" containerName="main" Jul 06 15:11:54.965094 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.965054 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" containerName="main" Jul 06 15:11:54.965364 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.965140 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="68d5dbdc-f12e-4242-b1e3-ab4381f1f1ea" containerName="maas-api" Jul 06 15:11:54.965364 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.965154 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c17f1d5-74d8-4b06-aac6-e2ac9603bc76" containerName="maas-api" Jul 06 15:11:54.965364 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.965166 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="98b4e3f8-cf5c-4a7b-ac6c-8305c92368b2" containerName="main" Jul 06 15:11:54.969415 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.969393 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:54.972379 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.972342 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2bspv\"/\"openshift-service-ca.crt\"" Jul 06 15:11:54.973330 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.973302 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2bspv\"/\"default-dockercfg-gn9kj\"" Jul 06 15:11:54.973461 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.973344 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2bspv\"/\"kube-root-ca.crt\"" Jul 06 15:11:54.980078 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:54.980054 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn"] Jul 06 15:11:55.090233 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.090169 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-lib-modules\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.090417 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.090245 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjkbv\" (UniqueName: \"kubernetes.io/projected/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-kube-api-access-wjkbv\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.090417 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.090271 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-proc\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.090417 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.090288 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-podres\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.090417 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.090351 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-sys\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.191555 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.191514 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-sys\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.191984 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.191577 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-lib-modules\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.191984 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.191599 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjkbv\" (UniqueName: \"kubernetes.io/projected/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-kube-api-access-wjkbv\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.191984 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.191628 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-proc\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.191984 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.191644 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-podres\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.191984 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.191647 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-sys\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.191984 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.191736 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-proc\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.191984 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.191759 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-podres\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.191984 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.191741 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-lib-modules\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.207710 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.207686 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjkbv\" (UniqueName: \"kubernetes.io/projected/e1a978bb-3c04-43d5-b0f1-4fa6632f9981-kube-api-access-wjkbv\") pod \"perf-node-gather-daemonset-2jnpn\" (UID: \"e1a978bb-3c04-43d5-b0f1-4fa6632f9981\") " pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.280008 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.279985 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:55.411990 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:55.411959 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn"] Jul 06 15:11:55.414466 ip-10-0-137-102 kubenswrapper[2569]: W0706 15:11:55.414434 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode1a978bb_3c04_43d5_b0f1_4fa6632f9981.slice/crio-64fd8ecf1847a7cdb9c907963680ee687c0d7e196f8256c981d1b3f802e7cc29 WatchSource:0}: Error finding container 64fd8ecf1847a7cdb9c907963680ee687c0d7e196f8256c981d1b3f802e7cc29: Status 404 returned error can't find the container with id 64fd8ecf1847a7cdb9c907963680ee687c0d7e196f8256c981d1b3f802e7cc29 Jul 06 15:11:56.352189 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:56.352149 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" event={"ID":"e1a978bb-3c04-43d5-b0f1-4fa6632f9981","Type":"ContainerStarted","Data":"f59f7acfec5e926783d82129196d7261a90978b1c4d39683a4eb0d2c552c9a94"} Jul 06 15:11:56.352652 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:56.352261 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:11:56.352652 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:56.352281 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" event={"ID":"e1a978bb-3c04-43d5-b0f1-4fa6632f9981","Type":"ContainerStarted","Data":"64fd8ecf1847a7cdb9c907963680ee687c0d7e196f8256c981d1b3f802e7cc29"} Jul 06 15:11:56.373739 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:56.373690 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" podStartSLOduration=2.373676408 podStartE2EDuration="2.373676408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:11:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:11:56.372857577 +0000 UTC m=+2584.013827507" watchObservedRunningTime="2026-07-06 15:11:56.373676408 +0000 UTC m=+2584.014646316" Jul 06 15:11:56.528020 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:56.527993 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/2.log" Jul 06 15:11:56.531894 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:56.531874 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-lsf6f_eafea937-7719-47d7-9240-e94089cdfcc4/console-operator/3.log" Jul 06 15:11:57.184235 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:57.184191 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-lbxkb_e90e93c3-77b0-4cc6-99c6-6c0ecf20eff9/download-server/0.log" Jul 06 15:11:57.899614 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:57.899581 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-brjmq_f9424d51-1719-4cfd-a85d-8163121ac834/volume-data-source-validator/0.log" Jul 06 15:11:59.395195 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:59.395171 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-czjlx_c27dfc49-d472-4046-a1ac-55191f07e7c1/dns/0.log" Jul 06 15:11:59.427479 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:59.427453 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-czjlx_c27dfc49-d472-4046-a1ac-55191f07e7c1/kube-rbac-proxy/0.log" Jul 06 15:11:59.604454 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:11:59.604426 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vvm5c_888d055b-85a7-4b48-becd-0f12b488f994/dns-node-resolver/0.log" Jul 06 15:12:00.295898 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:00.295870 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-99szs_1f8b3a09-7749-4094-a2b8-fe09f23b1d44/node-ca/0.log" Jul 06 15:12:01.727882 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:01.727849 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xzsp6_3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3/discovery/0.log" Jul 06 15:12:02.036913 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:02.036885 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fd947d8df-9l5bt_c56d1fb4-a9ab-41e7-b8ab-dd83500ec477/router/0.log" Jul 06 15:12:02.366088 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:02.366014 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2bspv/perf-node-gather-daemonset-2jnpn" Jul 06 15:12:02.953629 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:02.953599 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rnk6j_0dae0635-5d0a-47eb-994a-14f09891a285/serve-healthcheck-canary/0.log" Jul 06 15:12:03.646799 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:03.646770 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-5nw2p_86455cea-7f7b-47d6-a140-2809930847c8/insights-operator/0.log" Jul 06 15:12:03.647380 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:03.647358 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-5nw2p_86455cea-7f7b-47d6-a140-2809930847c8/insights-operator/1.log" Jul 06 15:12:03.794532 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:03.794502 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k5bxg_ea26ce84-0bf5-4578-a800-983f5646b2b7/kube-rbac-proxy/0.log" Jul 06 15:12:03.828645 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:03.828617 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k5bxg_ea26ce84-0bf5-4578-a800-983f5646b2b7/exporter/0.log" Jul 06 15:12:03.866069 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:03.866043 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k5bxg_ea26ce84-0bf5-4578-a800-983f5646b2b7/extractor/0.log" Jul 06 15:12:05.199149 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:05.199117 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xzsp6_3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3/discovery/0.log" Jul 06 15:12:05.422155 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:05.422079 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fd947d8df-9l5bt_c56d1fb4-a9ab-41e7-b8ab-dd83500ec477/router/0.log" Jul 06 15:12:06.829320 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:06.829282 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xzsp6_3ab8e800-bbb8-4b4b-baca-c8ebf85d90c3/discovery/0.log" Jul 06 15:12:07.241623 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:07.241594 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fd947d8df-9l5bt_c56d1fb4-a9ab-41e7-b8ab-dd83500ec477/router/0.log" Jul 06 15:12:07.295277 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:07.295247 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722485-f45qj_adf65a14-adcd-42a7-8ea8-ba0ed4977a02/cleanup/0.log" Jul 06 15:12:07.339194 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:07.339169 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722500-9vz9k_1b5338fe-97e2-400d-97a5-9bc218c3eea2/cleanup/0.log" Jul 06 15:12:07.750464 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:07.750431 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75545b5d88-4swgm_2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05/manager/0.log" Jul 06 15:12:07.795947 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:07.795920 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75545b5d88-4swgm_2b34ec27-1b0b-45d8-a7f9-bb3490a3dc05/copy-manifests/0.log" Jul 06 15:12:09.372200 ip-10-0-137-102 kubenswrapper[2569]: I0706 15:12:09.372166 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6689bc4cf6-h5mtg_825a13a3-1d3b-44af-b557-59d5b20d0d98/manager/0.log"