Jul 16 10:30:17.970614 ip-10-0-142-37 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 10:30:17.970628 ip-10-0-142-37 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 10:30:17.970638 ip-10-0-142-37 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 10:30:17.970976 ip-10-0-142-37 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 10:30:28.162790 ip-10-0-142-37 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 10:30:28.162807 ip-10-0-142-37 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 1513252cba2e47258798142a2fa8dfad -- Jul 16 10:32:25.209963 ip-10-0-142-37 systemd[1]: Starting Kubernetes Kubelet... Jul 16 10:32:25.585218 ip-10-0-142-37 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 10:32:25.585218 ip-10-0-142-37 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 10:32:25.585218 ip-10-0-142-37 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 10:32:25.585218 ip-10-0-142-37 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 10:32:25.585218 ip-10-0-142-37 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 10:32:25.587408 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.587255 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 10:32:25.591838 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591822 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:25.591838 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591838 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591842 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591845 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591848 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591852 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591854 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591857 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591860 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591865 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591869 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591872 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591875 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591878 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591882 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591885 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591889 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591892 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591895 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591898 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:25.591906 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591901 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591903 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591906 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591909 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591912 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591915 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591917 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591920 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591923 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591926 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591928 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591931 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591933 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591936 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591939 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591942 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591945 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591947 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591950 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591953 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:25.592371 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591955 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591958 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591960 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591963 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591965 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591967 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591970 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591972 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591978 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591981 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591983 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591986 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591989 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591992 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591994 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.591999 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592002 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592004 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592007 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592009 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:25.592858 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592012 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592016 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592020 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592023 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592026 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592029 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592032 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592035 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592037 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592042 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592045 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592048 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592050 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592053 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592056 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592058 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592061 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592063 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592066 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:25.593353 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592068 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592071 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592073 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592076 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592078 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592082 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592085 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592088 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592091 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592093 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592096 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592098 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592101 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592103 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592106 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592108 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592110 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592114 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592118 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:25.593817 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592120 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592854 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592861 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592865 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592867 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592870 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592874 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592877 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592879 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592882 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592884 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592887 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592889 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592892 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592894 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592897 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592900 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592902 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592905 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:25.594266 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592909 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592912 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592914 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592917 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592920 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592923 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592926 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592928 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592931 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592933 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592936 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592938 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592941 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592943 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592946 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592948 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592951 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592954 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592957 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592959 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:25.594754 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592963 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592966 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592968 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592971 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592973 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592975 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592978 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592982 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592985 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592988 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592990 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592993 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592996 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.592999 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593002 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593004 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593007 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593009 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593012 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593014 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:25.595231 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593017 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593019 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593022 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593027 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593030 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593034 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593087 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593092 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593094 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593097 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593100 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593103 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593107 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593111 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593113 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593117 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593120 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593124 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593127 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:25.595721 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593130 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593133 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593135 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593138 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593141 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593144 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593147 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593149 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593152 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593155 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593158 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593160 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593162 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593165 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593167 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593170 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593172 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593175 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593177 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593180 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:25.596200 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593182 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593184 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593601 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593607 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593610 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593613 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593616 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593620 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593625 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593628 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593631 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593634 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593637 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593640 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593642 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593645 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593647 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593650 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593653 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593655 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:25.596697 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593658 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593661 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593664 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593666 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593669 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593671 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593674 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593676 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593679 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593681 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593684 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593700 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593703 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593705 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593708 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593711 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593713 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593717 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593721 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593731 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:25.597199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593735 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593737 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593740 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593744 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593747 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593750 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593753 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593756 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593759 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593762 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593765 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593767 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593770 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593773 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593775 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593778 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593780 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593783 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593786 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593789 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:25.597750 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593792 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593794 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593797 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593799 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593801 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593804 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593807 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593810 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593814 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593816 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593819 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593822 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593826 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593829 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593832 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593834 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593838 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593840 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593843 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:25.598222 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593845 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593848 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593851 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593853 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593856 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593858 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593861 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593863 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593866 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593869 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593873 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593876 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593879 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593882 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593885 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593887 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593889 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593892 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593894 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:25.598746 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593897 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593899 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.593902 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.593978 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.593989 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.593998 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594005 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594019 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594024 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594033 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594038 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594041 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594044 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594047 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594051 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594054 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594058 2581 flags.go:64] FLAG: --cgroup-root="" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594061 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594064 2581 flags.go:64] FLAG: --client-ca-file="" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594066 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594069 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594074 2581 flags.go:64] FLAG: --cluster-domain="" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594077 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594082 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 10:32:25.599195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594085 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594089 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594093 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594096 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594099 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594103 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594106 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594109 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594112 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594115 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594118 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594122 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594125 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594128 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594131 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594135 2581 flags.go:64] FLAG: --enable-server="true" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594138 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594143 2581 flags.go:64] FLAG: --event-burst="100" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594146 2581 flags.go:64] FLAG: --event-qps="50" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594149 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594152 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594155 2581 flags.go:64] FLAG: --eviction-hard="" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594159 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594162 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594165 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 10:32:25.599796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594169 2581 flags.go:64] FLAG: --eviction-soft="" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594171 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594175 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594178 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594181 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594184 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594188 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594191 2581 flags.go:64] FLAG: --feature-gates="" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594195 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594198 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594201 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594204 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594207 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594210 2581 flags.go:64] FLAG: --help="false" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594213 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-142-37.ec2.internal" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594216 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594219 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594222 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594225 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594228 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594231 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594234 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594237 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 10:32:25.600398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594240 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594245 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594248 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594251 2581 flags.go:64] FLAG: --kube-reserved="" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594254 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594256 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594259 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594262 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594265 2581 flags.go:64] FLAG: --lock-file="" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594268 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594271 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594274 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594280 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594282 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594285 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594289 2581 flags.go:64] FLAG: --logging-format="text" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594292 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594296 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594298 2581 flags.go:64] FLAG: --manifest-url="" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594301 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594306 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594309 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594313 2581 flags.go:64] FLAG: --max-pods="110" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594316 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594319 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 10:32:25.601001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594322 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594325 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594328 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594331 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594333 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594340 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594343 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594346 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594352 2581 flags.go:64] FLAG: --pod-cidr="" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594356 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594361 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594364 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594368 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594371 2581 flags.go:64] FLAG: --port="10250" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594373 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594376 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b9d9390f2014d722" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594379 2581 flags.go:64] FLAG: --qos-reserved="" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594382 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594386 2581 flags.go:64] FLAG: --register-node="true" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594389 2581 flags.go:64] FLAG: --register-with-taints="" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594392 2581 flags.go:64] FLAG: --registry-burst="10" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594395 2581 flags.go:64] FLAG: --registry-qps="5" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594399 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594402 2581 flags.go:64] FLAG: --reserved-memory="" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594406 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 10:32:25.601625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594409 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594412 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594415 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594417 2581 flags.go:64] FLAG: --runonce="false" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594420 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594423 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594426 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594429 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594432 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594435 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594438 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594441 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594444 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594447 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594450 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594453 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594458 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594461 2581 flags.go:64] FLAG: --system-cgroups="" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594464 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594469 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594472 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594475 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594480 2581 flags.go:64] FLAG: --tls-min-version="" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594482 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594485 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 10:32:25.602292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594488 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594505 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594509 2581 flags.go:64] FLAG: --v="2" Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594514 2581 flags.go:64] FLAG: --version="false" Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594521 2581 flags.go:64] FLAG: --vmodule="" Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594525 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.594528 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594656 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594660 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594663 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594666 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594669 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594671 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594674 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594678 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594682 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594700 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594703 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594706 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594709 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594712 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:25.602984 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594715 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594718 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594722 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594726 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594729 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594732 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594735 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594737 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594740 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594743 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594745 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594748 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594750 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594753 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594755 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594760 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594763 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594766 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594768 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594771 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:25.603802 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594773 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594776 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594778 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594781 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594784 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594786 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594789 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594791 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594794 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594797 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594799 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594802 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594804 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594806 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594810 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594813 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594815 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594819 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594823 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:25.604438 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594827 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594830 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594832 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594835 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594838 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594841 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594844 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594847 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594852 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594855 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594858 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594861 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594863 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594866 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594869 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594872 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594875 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594878 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594880 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594883 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:25.604920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594886 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594888 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594891 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594893 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594896 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594899 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594902 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594906 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594908 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594911 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594914 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594917 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594920 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594922 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594925 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594927 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594930 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594932 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594935 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594937 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:25.605449 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594941 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594944 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594946 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594949 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594951 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.594954 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.595461 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.602105 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.602120 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.602142 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.602151 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602208 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602214 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602218 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:25.605947 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602222 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602225 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602228 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602230 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602233 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602236 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602240 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602242 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602245 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602248 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602251 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602254 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602256 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602259 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602261 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602264 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602267 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602269 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602272 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602275 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:25.606313 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602278 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602280 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602283 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602286 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602289 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602291 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602294 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602297 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602300 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602303 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602306 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602308 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602311 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602314 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602316 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602319 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602321 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602324 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602327 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602330 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:25.606885 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602332 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602334 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602337 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602340 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602343 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602345 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602348 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602350 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602353 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602356 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602358 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602361 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602363 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602366 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602369 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602372 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602375 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602377 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602380 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:25.607333 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602383 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602385 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602387 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602390 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602393 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602395 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602398 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602401 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602404 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602408 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602411 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602414 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602417 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602419 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602422 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602425 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602429 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602432 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602435 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:25.607806 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602438 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602440 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602444 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602447 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602450 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602453 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602456 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602458 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602461 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602463 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602466 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602468 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602471 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602473 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602476 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602479 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602481 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:25.608262 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602484 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.602489 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602611 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602616 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602619 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602622 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602625 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602628 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602631 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602633 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602639 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602643 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602645 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602649 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 10:32:25.608656 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602653 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602656 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602659 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602662 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602665 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602667 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602670 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602673 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602675 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602677 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602680 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602682 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602701 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602704 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602707 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602710 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602713 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602715 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602718 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 10:32:25.609011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602722 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602725 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602728 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602732 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602735 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602739 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602741 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602744 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602747 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602750 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602753 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602755 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602758 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602761 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602763 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602766 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602768 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602771 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602774 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 10:32:25.609478 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602776 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602779 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602781 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602784 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602786 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602789 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602792 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602796 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602799 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602802 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602804 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602807 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602810 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602813 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602815 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602818 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602820 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602823 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602825 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602828 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 10:32:25.609959 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602831 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602834 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602836 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602839 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602841 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602844 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602846 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602849 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602852 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602854 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602857 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602859 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602862 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602864 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602867 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602869 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602872 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602874 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602877 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602879 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 10:32:25.610452 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602882 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 10:32:25.610956 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602884 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 10:32:25.610956 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602887 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 10:32:25.610956 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602889 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 10:32:25.610956 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602892 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 10:32:25.610956 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602895 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 10:32:25.610956 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602898 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 10:32:25.610956 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602901 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 10:32:25.610956 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:25.602903 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 10:32:25.610956 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.602909 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 10:32:25.610956 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.605515 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 10:32:25.610956 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.609728 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 10:32:25.611224 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.611086 2581 server.go:1019] "Starting client certificate rotation" Jul 16 10:32:25.611224 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.611186 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 10:32:25.611277 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.611224 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 10:32:25.613540 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.613517 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 10:32:25.617181 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.617166 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 10:32:25.627233 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.627210 2581 log.go:25] "Validated CRI v1 runtime API" Jul 16 10:32:25.630617 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.630603 2581 log.go:25] "Validated CRI v1 image API" Jul 16 10:32:25.631472 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.631455 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 10:32:25.633885 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.633857 2581 fs.go:135] Filesystem UUIDs: map[6b499a6c-be32-4ccf-af84-14c75fca8fdf:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 abfc81b0-e992-47c5-8d8d-922e9b41f63c:/dev/nvme0n1p3] Jul 16 10:32:25.633955 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.633881 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 10:32:25.639335 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.638897 2581 manager.go:216] Machine: {Timestamp:2026-07-16 10:32:25.638407328 +0000 UTC m=+0.317672303 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3172913 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27194b749bb4b7cd02b230ea2b3f00 SystemUUID:ec27194b-749b-b4b7-cd02-b230ea2b3f00 BootID:1513252c-ba2e-4725-8798-142a2fa8dfad Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1c:4b:1f:c6:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1c:4b:1f:c6:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:5d:32:96:c7:ec Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 10:32:25.639335 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.639322 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 10:32:25.639533 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.639452 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 10:32:25.641204 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.641182 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 10:32:25.643025 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.642837 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 10:32:25.643158 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.643032 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-37.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 10:32:25.643934 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.643923 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 10:32:25.643991 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.643937 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 10:32:25.643991 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.643944 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 10:32:25.644567 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.644557 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 10:32:25.645369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.645355 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 10:32:25.648181 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.648170 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 16 10:32:25.648306 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.648297 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 10:32:25.648341 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.648326 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 16 10:32:25.648341 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.648339 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 10:32:25.648394 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.648350 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 10:32:25.648394 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.648360 2581 kubelet.go:398] "Adding apiserver pod source" Jul 16 10:32:25.648394 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.648371 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 10:32:25.648714 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.648702 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 10:32:25.648771 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.648720 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 10:32:25.650880 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.650863 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 10:32:25.652116 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652101 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 10:32:25.652202 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652139 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 10:32:25.652801 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652787 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 10:32:25.652878 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652807 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 10:32:25.652878 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652817 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 10:32:25.652878 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652825 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 10:32:25.652878 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652838 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 10:32:25.652878 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652847 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 10:32:25.652878 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652859 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 10:32:25.652878 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652867 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 10:32:25.652878 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652879 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 10:32:25.653108 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652889 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 10:32:25.653108 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652907 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 10:32:25.653108 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.652916 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 10:32:25.653607 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.653594 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 10:32:25.653649 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.653610 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 10:32:25.656730 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.656713 2581 server.go:1268] "Started kubelet" Jul 16 10:32:25.656806 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.656777 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 10:32:25.656913 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.656870 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 10:32:25.656963 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.656936 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 10:32:25.658002 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.657982 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 10:32:25.658732 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.658704 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 10:32:25.658732 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.658709 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 10:32:25.658886 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.658817 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-37.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 10:32:25.658830 ip-10-0-142-37 systemd[1]: Started Kubernetes Kubelet. Jul 16 10:32:25.659569 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.659527 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 16 10:32:25.661819 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.661802 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 10:32:25.661819 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.661813 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 10:32:25.662020 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.662004 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:25.662164 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.661177 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-37.ec2.internal.18c2be9417d8e9b5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-37.ec2.internal,UID:ip-10-0-142-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-37.ec2.internal,},FirstTimestamp:2026-07-16 10:32:25.656666549 +0000 UTC m=+0.335931519,LastTimestamp:2026-07-16 10:32:25.656666549 +0000 UTC m=+0.335931519,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-37.ec2.internal,}" Jul 16 10:32:25.662542 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.662525 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 10:32:25.662663 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.662606 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 10:32:25.662663 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.662659 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 10:32:25.662821 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.662794 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 16 10:32:25.662821 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.662802 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 16 10:32:25.662978 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.662959 2581 factory.go:55] Registering systemd factory Jul 16 10:32:25.662978 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.662978 2581 factory.go:223] Registration of the systemd container factory successfully Jul 16 10:32:25.663204 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.663189 2581 factory.go:153] Registering CRI-O factory Jul 16 10:32:25.663278 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.663208 2581 factory.go:223] Registration of the crio container factory successfully Jul 16 10:32:25.663278 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.663254 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 10:32:25.663362 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.663283 2581 factory.go:103] Registering Raw factory Jul 16 10:32:25.663362 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.663297 2581 manager.go:1196] Started watching for new ooms in manager Jul 16 10:32:25.663438 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.663409 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 10:32:25.664019 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.664005 2581 manager.go:349] Starting recovery of all containers Jul 16 10:32:25.664914 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.664879 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 10:32:25.665010 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.664980 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 10:32:25.669899 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.669856 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 10:32:25.675580 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.675545 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nxbm5" Jul 16 10:32:25.677868 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.677720 2581 manager.go:354] Recovery completed Jul 16 10:32:25.682000 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.681988 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:25.683047 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.683028 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nxbm5" Jul 16 10:32:25.684069 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.684053 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:25.684148 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.684085 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:25.684148 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.684100 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:25.684619 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.684606 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:25.684681 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.684606 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 10:32:25.684681 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.684669 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 10:32:25.684773 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.684703 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 16 10:32:25.685322 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.685306 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:25.685374 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.685332 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:25.685374 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.685342 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:25.685784 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.685678 2581 policy_none.go:49] "None policy: Start" Jul 16 10:32:25.685784 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.685727 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 10:32:25.685784 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.685738 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 10:32:25.686376 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.686249 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-37.ec2.internal.18c2be94197b0905 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-37.ec2.internal,UID:ip-10-0-142-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-37.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-37.ec2.internal,},FirstTimestamp:2026-07-16 10:32:25.684068613 +0000 UTC m=+0.363333607,LastTimestamp:2026-07-16 10:32:25.684068613 +0000 UTC m=+0.363333607,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-37.ec2.internal,}" Jul 16 10:32:25.686758 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.686748 2581 policy_none.go:47] "Start" Jul 16 10:32:25.723271 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.723255 2581 manager.go:337] "Starting Device Plugin manager" Jul 16 10:32:25.723352 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.723294 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 10:32:25.723352 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.723306 2581 server.go:87] "Starting device plugin registration server" Jul 16 10:32:25.723567 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.723556 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 10:32:25.723643 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.723569 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 10:32:25.723736 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.723651 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 10:32:25.723784 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.723746 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 10:32:25.723784 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.723754 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 10:32:25.724285 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.724266 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 10:32:25.724344 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.724321 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:25.799540 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.799508 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 10:32:25.800712 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.799550 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 10:32:25.800712 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.799570 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 10:32:25.800712 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.799600 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 10:32:25.803247 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.803232 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 10:32:25.824281 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.824262 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:25.825167 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.825144 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:25.825300 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.825179 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:25.825300 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.825192 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:25.825300 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.825222 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-37.ec2.internal" Jul 16 10:32:25.847573 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.847522 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-37.ec2.internal" Jul 16 10:32:25.847573 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.847548 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-37.ec2.internal\": node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:25.882503 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.882472 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:25.900360 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.900335 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-37.ec2.internal"] Jul 16 10:32:25.900470 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.900413 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:25.901906 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.901891 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:25.902002 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.901922 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:25.902002 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.901936 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:25.903033 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.903019 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:25.903151 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.903136 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" Jul 16 10:32:25.903194 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.903165 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:25.903745 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.903727 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:25.903843 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.903752 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:25.903843 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.903761 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:25.903843 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.903764 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:25.903843 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.903785 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:25.903843 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.903798 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:25.904817 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.904800 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-37.ec2.internal" Jul 16 10:32:25.904895 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.904828 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 10:32:25.905435 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.905417 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientMemory" Jul 16 10:32:25.905506 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.905442 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 10:32:25.905506 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.905451 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeHasSufficientPID" Jul 16 10:32:25.922209 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.922192 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-37.ec2.internal\" not found" node="ip-10-0-142-37.ec2.internal" Jul 16 10:32:25.926355 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.926339 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-37.ec2.internal\" not found" node="ip-10-0-142-37.ec2.internal" Jul 16 10:32:25.964205 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.964152 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/446caf796df52d1b266f588acb053144-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal\" (UID: \"446caf796df52d1b266f588acb053144\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" Jul 16 10:32:25.964262 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.964211 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/446caf796df52d1b266f588acb053144-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal\" (UID: \"446caf796df52d1b266f588acb053144\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" Jul 16 10:32:25.964262 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:25.964229 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/26a59dee568ae6e70ff513d52676f317-config\") pod \"kube-apiserver-proxy-ip-10-0-142-37.ec2.internal\" (UID: \"26a59dee568ae6e70ff513d52676f317\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-37.ec2.internal" Jul 16 10:32:25.982921 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:25.982896 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:26.064489 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.064463 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/446caf796df52d1b266f588acb053144-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal\" (UID: \"446caf796df52d1b266f588acb053144\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" Jul 16 10:32:26.064489 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.064491 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/26a59dee568ae6e70ff513d52676f317-config\") pod \"kube-apiserver-proxy-ip-10-0-142-37.ec2.internal\" (UID: \"26a59dee568ae6e70ff513d52676f317\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-37.ec2.internal" Jul 16 10:32:26.064631 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.064510 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/446caf796df52d1b266f588acb053144-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal\" (UID: \"446caf796df52d1b266f588acb053144\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" Jul 16 10:32:26.064631 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.064536 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/446caf796df52d1b266f588acb053144-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal\" (UID: \"446caf796df52d1b266f588acb053144\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" Jul 16 10:32:26.064631 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.064556 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/446caf796df52d1b266f588acb053144-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal\" (UID: \"446caf796df52d1b266f588acb053144\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" Jul 16 10:32:26.064631 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.064564 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/26a59dee568ae6e70ff513d52676f317-config\") pod \"kube-apiserver-proxy-ip-10-0-142-37.ec2.internal\" (UID: \"26a59dee568ae6e70ff513d52676f317\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-37.ec2.internal" Jul 16 10:32:26.083911 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:26.083891 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:26.184479 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:26.184389 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:26.224575 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.224551 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" Jul 16 10:32:26.229526 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.229510 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-37.ec2.internal" Jul 16 10:32:26.285276 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:26.285227 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:26.385783 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:26.385754 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:26.486295 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:26.486209 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:26.586760 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:26.586734 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:26.610898 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.610880 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 10:32:26.611020 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.611003 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 10:32:26.630364 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.630341 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 10:32:26.686757 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.686724 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 10:27:26 +0000 UTC" deadline="2028-02-23 05:26:07.992880583 +0000 UTC" Jul 16 10:32:26.686757 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.686748 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14082h53m41.306134771s" Jul 16 10:32:26.686909 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:26.686771 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:26.703811 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.703789 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 10:32:26.787593 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:26.787570 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-37.ec2.internal\" not found" Jul 16 10:32:26.827562 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.827539 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 10:32:26.862836 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.862813 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" Jul 16 10:32:26.870617 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:26.870580 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod446caf796df52d1b266f588acb053144.slice/crio-004965d2bc0c06042feac3ed225d5e8aa74f162560090e0ddc694a8478c21f23 WatchSource:0}: Error finding container 004965d2bc0c06042feac3ed225d5e8aa74f162560090e0ddc694a8478c21f23: Status 404 returned error can't find the container with id 004965d2bc0c06042feac3ed225d5e8aa74f162560090e0ddc694a8478c21f23 Jul 16 10:32:26.870974 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:26.870957 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a59dee568ae6e70ff513d52676f317.slice/crio-a9df066f9eef53624c0dc07f2a7373b8b07a82b9d25892aee0e9b69493d9ad24 WatchSource:0}: Error finding container a9df066f9eef53624c0dc07f2a7373b8b07a82b9d25892aee0e9b69493d9ad24: Status 404 returned error can't find the container with id a9df066f9eef53624c0dc07f2a7373b8b07a82b9d25892aee0e9b69493d9ad24 Jul 16 10:32:26.874794 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.874781 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 10:32:26.876012 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.875999 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 10:32:26.877369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.877234 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-37.ec2.internal" Jul 16 10:32:26.887611 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:26.887596 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 10:32:27.649550 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.649466 2581 apiserver.go:52] "Watching apiserver" Jul 16 10:32:27.659003 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.658972 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 10:32:27.661286 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.661247 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-37.ec2.internal","openshift-cluster-node-tuning-operator/tuned-p68l8","openshift-multus/multus-additional-cni-plugins-7b4t8","openshift-multus/multus-ztj49","openshift-ovn-kubernetes/ovnkube-node-4n6zq","kube-system/global-pull-secret-syncer-82c65","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s","openshift-dns/node-resolver-mtklw","openshift-image-registry/node-ca-jkm76","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal","openshift-multus/network-metrics-daemon-rdp5f","openshift-network-diagnostics/network-check-target-lnndv","openshift-network-operator/iptables-alerter-jqlcl","kube-system/konnectivity-agent-zs5vl"] Jul 16 10:32:27.662844 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.662823 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 10:32:27.662950 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.662935 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.664549 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.664531 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jqlcl" Jul 16 10:32:27.665912 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.665868 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.666021 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.666002 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c69cb\"" Jul 16 10:32:27.666404 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.666383 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 10:32:27.666551 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.666456 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 10:32:27.667189 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.667163 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 10:32:27.667287 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.667191 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:32:27.667370 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.667298 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.667370 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.667324 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 10:32:27.668772 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.668754 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.670272 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.670252 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:27.670365 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:27.670343 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:27.671571 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.671296 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-m9vt4\"" Jul 16 10:32:27.671571 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.671373 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 10:32:27.671571 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.671418 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kjhsp\"" Jul 16 10:32:27.671571 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.671421 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 10:32:27.671571 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.671531 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 10:32:27.672146 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.671658 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jkm76" Jul 16 10:32:27.672146 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.671786 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.672033 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.672211 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.672309 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hrn99\"" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.672339 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.672415 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673202 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673266 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sn4g7\"" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.672914 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.672882 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-registration-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673317 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673356 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-sys-fs\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673388 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/19ca7547-d371-4c98-b81d-9b0812440419-iptables-alerter-script\") pod \"iptables-alerter-jqlcl\" (UID: \"19ca7547-d371-4c98-b81d-9b0812440419\") " pod="openshift-network-operator/iptables-alerter-jqlcl" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673414 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgzdn\" (UniqueName: \"kubernetes.io/projected/19ca7547-d371-4c98-b81d-9b0812440419-kube-api-access-fgzdn\") pod \"iptables-alerter-jqlcl\" (UID: \"19ca7547-d371-4c98-b81d-9b0812440419\") " pod="openshift-network-operator/iptables-alerter-jqlcl" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673440 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673463 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-socket-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673472 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 10:32:27.673476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673486 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-device-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.674316 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673528 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-etc-selinux\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.674316 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673577 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xts5m\" (UniqueName: \"kubernetes.io/projected/eded4b36-847f-4ecc-9f18-4e57499337bf-kube-api-access-xts5m\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.674316 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673592 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 10:32:27.674316 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673600 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 10:32:27.674316 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.673634 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19ca7547-d371-4c98-b81d-9b0812440419-host-slash\") pod \"iptables-alerter-jqlcl\" (UID: \"19ca7547-d371-4c98-b81d-9b0812440419\") " pod="openshift-network-operator/iptables-alerter-jqlcl" Jul 16 10:32:27.674563 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.674341 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.675200 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.675183 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 10:32:27.675873 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.675813 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 10:32:27.676484 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.676462 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mtklw" Jul 16 10:32:27.677959 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.677936 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 10:32:27.678481 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.678456 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 10:32:27.678749 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.678731 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:27.679167 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:27.679139 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:27.679999 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.679931 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rkxvk\"" Jul 16 10:32:27.681629 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.680791 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 10:32:27.681629 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.680952 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 10:32:27.681629 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.681048 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 10:32:27.681629 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.681089 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:32:27.681629 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.681379 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fqf46\"" Jul 16 10:32:27.681962 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.681758 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jtz2h\"" Jul 16 10:32:27.681962 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.681823 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:27.681962 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:27.681894 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:27.681962 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.681915 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 10:32:27.682271 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.682250 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-crxpx\"" Jul 16 10:32:27.682947 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.682851 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 10:32:27.705325 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.705219 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-26snb" Jul 16 10:32:27.715110 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.715090 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-26snb" Jul 16 10:32:27.764127 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.764104 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 10:32:27.774274 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774250 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-os-release\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.774274 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774278 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-kubelet\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.774468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774296 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-run-systemd\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.774468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774312 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f72393fd-77d3-4b0d-85c6-310f88907af1-env-overrides\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.774468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774331 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6v47\" (UniqueName: \"kubernetes.io/projected/f72393fd-77d3-4b0d-85c6-310f88907af1-kube-api-access-r6v47\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.774468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774362 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:27.774468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774380 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-system-cni-dir\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.774468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774394 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-systemd\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.774468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774409 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-run-ovn\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.774468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/19ca7547-d371-4c98-b81d-9b0812440419-iptables-alerter-script\") pod \"iptables-alerter-jqlcl\" (UID: \"19ca7547-d371-4c98-b81d-9b0812440419\") " pod="openshift-network-operator/iptables-alerter-jqlcl" Jul 16 10:32:27.774468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774453 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgzdn\" (UniqueName: \"kubernetes.io/projected/19ca7547-d371-4c98-b81d-9b0812440419-kube-api-access-fgzdn\") pod \"iptables-alerter-jqlcl\" (UID: \"19ca7547-d371-4c98-b81d-9b0812440419\") " pod="openshift-network-operator/iptables-alerter-jqlcl" Jul 16 10:32:27.774468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774470 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.774879 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774492 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-sys\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.774879 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774538 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-tuned\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.774879 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774582 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fda8e68-706b-4c7d-998d-ba89897e6457-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.774879 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774620 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fae8c27e-c7eb-4f51-af72-4f0a88996042-tmp\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.774879 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774669 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:27.774879 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774730 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-run-netns\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.774879 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774763 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-var-lib-cni-multus\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.774879 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774837 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-cnibin\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774901 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk7ng\" (UniqueName: \"kubernetes.io/projected/0cb30e43-e55f-4dec-af3f-39960a5946cb-kube-api-access-fk7ng\") pod \"node-resolver-mtklw\" (UID: \"0cb30e43-e55f-4dec-af3f-39960a5946cb\") " pod="openshift-dns/node-resolver-mtklw" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774953 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19ca7547-d371-4c98-b81d-9b0812440419-host-slash\") pod \"iptables-alerter-jqlcl\" (UID: \"19ca7547-d371-4c98-b81d-9b0812440419\") " pod="openshift-network-operator/iptables-alerter-jqlcl" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.774983 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-registration-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775021 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-run\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775025 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19ca7547-d371-4c98-b81d-9b0812440419-host-slash\") pod \"iptables-alerter-jqlcl\" (UID: \"19ca7547-d371-4c98-b81d-9b0812440419\") " pod="openshift-network-operator/iptables-alerter-jqlcl" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775039 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/19ca7547-d371-4c98-b81d-9b0812440419-iptables-alerter-script\") pod \"iptables-alerter-jqlcl\" (UID: \"19ca7547-d371-4c98-b81d-9b0812440419\") " pod="openshift-network-operator/iptables-alerter-jqlcl" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775048 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-conf-dir\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775077 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-daemon-config\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775097 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-registration-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775102 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-cni-bin\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775161 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-device-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775192 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-var-lib-cni-bin\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775221 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-device-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.775229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775221 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3283c783-546e-4a74-8c2c-40ab4dd734e6-cni-binary-copy\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775266 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-run-openvswitch\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775303 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775333 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24b769d5-b98a-45d0-97fa-e67a68008449-kubelet-config\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775364 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-etc-selinux\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775413 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-os-release\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775464 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fda8e68-706b-4c7d-998d-ba89897e6457-cni-binary-copy\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775504 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fda8e68-706b-4c7d-998d-ba89897e6457-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775529 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-sysconfig\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775536 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-etc-selinux\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775546 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l47zt\" (UniqueName: \"kubernetes.io/projected/fae8c27e-c7eb-4f51-af72-4f0a88996042-kube-api-access-l47zt\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775586 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0cb30e43-e55f-4dec-af3f-39960a5946cb-hosts-file\") pod \"node-resolver-mtklw\" (UID: \"0cb30e43-e55f-4dec-af3f-39960a5946cb\") " pod="openshift-dns/node-resolver-mtklw" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775619 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-system-cni-dir\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775647 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-sys-fs\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775676 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e086493-640f-43e4-a8bf-cc5f03d52abe-agent-certs\") pod \"konnectivity-agent-zs5vl\" (UID: \"0e086493-640f-43e4-a8bf-cc5f03d52abe\") " pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775686 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-sys-fs\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.775751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775729 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e086493-640f-43e4-a8bf-cc5f03d52abe-konnectivity-ca\") pod \"konnectivity-agent-zs5vl\" (UID: \"0e086493-640f-43e4-a8bf-cc5f03d52abe\") " pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775746 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-var-lib-kubelet\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775762 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-run-multus-certs\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775776 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-systemd-units\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775796 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-run-ovn-kubernetes\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775825 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f72393fd-77d3-4b0d-85c6-310f88907af1-ovn-node-metrics-cert\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775853 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xts5m\" (UniqueName: \"kubernetes.io/projected/eded4b36-847f-4ecc-9f18-4e57499337bf-kube-api-access-xts5m\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775895 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-sysctl-d\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775925 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cb30e43-e55f-4dec-af3f-39960a5946cb-tmp-dir\") pod \"node-resolver-mtklw\" (UID: \"0cb30e43-e55f-4dec-af3f-39960a5946cb\") " pod="openshift-dns/node-resolver-mtklw" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775951 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-socket-dir-parent\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.775972 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-run-netns\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776012 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-log-socket\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776047 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f72393fd-77d3-4b0d-85c6-310f88907af1-ovnkube-config\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776096 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-etc-kubernetes\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776130 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqjqr\" (UniqueName: \"kubernetes.io/projected/3283c783-546e-4a74-8c2c-40ab4dd734e6-kube-api-access-zqjqr\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776157 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-cni-netd\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f72393fd-77d3-4b0d-85c6-310f88907af1-ovnkube-script-lib\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.776369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24b769d5-b98a-45d0-97fa-e67a68008449-dbus\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776232 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-cni-dir\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776256 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-cnibin\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776278 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880-host\") pod \"node-ca-jkm76\" (UID: \"b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880\") " pod="openshift-image-registry/node-ca-jkm76" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776303 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-socket-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776330 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5sp8\" (UniqueName: \"kubernetes.io/projected/0fda8e68-706b-4c7d-998d-ba89897e6457-kube-api-access-p5sp8\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776355 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-kubernetes\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776378 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-sysctl-conf\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776416 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-var-lib-kubelet\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776440 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880-serviceca\") pod \"node-ca-jkm76\" (UID: \"b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880\") " pod="openshift-image-registry/node-ca-jkm76" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776445 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-socket-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776527 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-hostroot\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776554 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-etc-openvswitch\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776581 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776605 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x68nl\" (UniqueName: \"kubernetes.io/projected/b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880-kube-api-access-x68nl\") pod \"node-ca-jkm76\" (UID: \"b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880\") " pod="openshift-image-registry/node-ca-jkm76" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776636 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vwqw\" (UniqueName: \"kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw\") pod \"network-check-target-lnndv\" (UID: \"a79fe0b6-6a92-4396-b856-d50f5c5383a3\") " pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776653 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eded4b36-847f-4ecc-9f18-4e57499337bf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.777058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776667 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-modprobe-d\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.777552 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776694 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-lib-modules\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.777552 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776716 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-var-lib-openvswitch\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.777552 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776766 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-host\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.777552 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776793 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-run-k8s-cni-cncf-io\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.777552 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776811 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-slash\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.777552 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776826 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-node-log\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.777552 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.776844 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ljdp\" (UniqueName: \"kubernetes.io/projected/a6cb6750-2018-4424-b782-201c673ee132-kube-api-access-2ljdp\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:27.782404 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.782380 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 10:32:27.785547 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.785529 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgzdn\" (UniqueName: \"kubernetes.io/projected/19ca7547-d371-4c98-b81d-9b0812440419-kube-api-access-fgzdn\") pod \"iptables-alerter-jqlcl\" (UID: \"19ca7547-d371-4c98-b81d-9b0812440419\") " pod="openshift-network-operator/iptables-alerter-jqlcl" Jul 16 10:32:27.785658 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.785636 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xts5m\" (UniqueName: \"kubernetes.io/projected/eded4b36-847f-4ecc-9f18-4e57499337bf-kube-api-access-xts5m\") pod \"aws-ebs-csi-driver-node-h5p9s\" (UID: \"eded4b36-847f-4ecc-9f18-4e57499337bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.804003 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.803954 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-37.ec2.internal" event={"ID":"26a59dee568ae6e70ff513d52676f317","Type":"ContainerStarted","Data":"a9df066f9eef53624c0dc07f2a7373b8b07a82b9d25892aee0e9b69493d9ad24"} Jul 16 10:32:27.805188 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.805167 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" event={"ID":"446caf796df52d1b266f588acb053144","Type":"ContainerStarted","Data":"004965d2bc0c06042feac3ed225d5e8aa74f162560090e0ddc694a8478c21f23"} Jul 16 10:32:27.877830 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.877799 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-cnibin\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.877960 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.877837 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880-host\") pod \"node-ca-jkm76\" (UID: \"b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880\") " pod="openshift-image-registry/node-ca-jkm76" Jul 16 10:32:27.878022 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.877862 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5sp8\" (UniqueName: \"kubernetes.io/projected/0fda8e68-706b-4c7d-998d-ba89897e6457-kube-api-access-p5sp8\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.878146 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878115 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880-host\") pod \"node-ca-jkm76\" (UID: \"b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880\") " pod="openshift-image-registry/node-ca-jkm76" Jul 16 10:32:27.878227 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878169 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-kubernetes\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.878227 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878208 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-sysctl-conf\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.878326 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878240 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-var-lib-kubelet\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.878326 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878276 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-cnibin\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.878326 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878290 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880-serviceca\") pod \"node-ca-jkm76\" (UID: \"b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880\") " pod="openshift-image-registry/node-ca-jkm76" Jul 16 10:32:27.878459 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-hostroot\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.878459 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878377 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-etc-openvswitch\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.878459 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878401 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x68nl\" (UniqueName: \"kubernetes.io/projected/b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880-kube-api-access-x68nl\") pod \"node-ca-jkm76\" (UID: \"b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880\") " pod="openshift-image-registry/node-ca-jkm76" Jul 16 10:32:27.878596 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878475 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vwqw\" (UniqueName: \"kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw\") pod \"network-check-target-lnndv\" (UID: \"a79fe0b6-6a92-4396-b856-d50f5c5383a3\") " pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:27.878596 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878482 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-hostroot\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.878596 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878529 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-modprobe-d\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.878596 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878554 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-lib-modules\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.878596 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878580 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-var-lib-openvswitch\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.878859 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878582 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-kubernetes\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.878859 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878609 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-host\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.878859 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878668 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-host\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.878859 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878528 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-etc-openvswitch\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.878859 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878728 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-var-lib-kubelet\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.878859 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878797 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-modprobe-d\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.878859 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878824 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-run-k8s-cni-cncf-io\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.878859 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878841 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-lib-modules\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878867 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-slash\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878898 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-run-k8s-cni-cncf-io\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878897 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-node-log\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878924 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-var-lib-openvswitch\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878945 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-node-log\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878928 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-sysctl-conf\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.878945 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ljdp\" (UniqueName: \"kubernetes.io/projected/a6cb6750-2018-4424-b782-201c673ee132-kube-api-access-2ljdp\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.879002 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-os-release\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.879035 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-kubelet\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.879066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-run-systemd\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.879091 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f72393fd-77d3-4b0d-85c6-310f88907af1-env-overrides\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.879143 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6v47\" (UniqueName: \"kubernetes.io/projected/f72393fd-77d3-4b0d-85c6-310f88907af1-kube-api-access-r6v47\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.879177 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:27.879214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.879210 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-system-cni-dir\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.879843 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.879276 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-run-systemd\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.879843 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.879294 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-slash\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.880275 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880174 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-system-cni-dir\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.880275 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880180 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-systemd\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.880275 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880226 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-run-ovn\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.880275 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880240 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-systemd\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.880508 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880288 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.880508 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880323 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-sys\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.880508 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880359 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-tuned\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.880508 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880388 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fda8e68-706b-4c7d-998d-ba89897e6457-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.880508 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880444 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fae8c27e-c7eb-4f51-af72-4f0a88996042-tmp\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.880508 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880481 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880513 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-run-netns\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880544 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-var-lib-cni-multus\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880538 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880579 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-cnibin\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880622 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-run-ovn\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880647 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-sys\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880661 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk7ng\" (UniqueName: \"kubernetes.io/projected/0cb30e43-e55f-4dec-af3f-39960a5946cb-kube-api-access-fk7ng\") pod \"node-resolver-mtklw\" (UID: \"0cb30e43-e55f-4dec-af3f-39960a5946cb\") " pod="openshift-dns/node-resolver-mtklw" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880698 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-run\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-conf-dir\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880775 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-daemon-config\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880806 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-cni-bin\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880838 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-var-lib-cni-bin\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.880882 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880869 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3283c783-546e-4a74-8c2c-40ab4dd734e6-cni-binary-copy\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880913 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-run-openvswitch\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880961 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881024 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24b769d5-b98a-45d0-97fa-e67a68008449-kubelet-config\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881087 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-os-release\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881128 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fda8e68-706b-4c7d-998d-ba89897e6457-cni-binary-copy\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881175 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fda8e68-706b-4c7d-998d-ba89897e6457-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881204 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-sysconfig\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881235 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l47zt\" (UniqueName: \"kubernetes.io/projected/fae8c27e-c7eb-4f51-af72-4f0a88996042-kube-api-access-l47zt\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881265 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0cb30e43-e55f-4dec-af3f-39960a5946cb-hosts-file\") pod \"node-resolver-mtklw\" (UID: \"0cb30e43-e55f-4dec-af3f-39960a5946cb\") " pod="openshift-dns/node-resolver-mtklw" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881303 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-system-cni-dir\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881348 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e086493-640f-43e4-a8bf-cc5f03d52abe-agent-certs\") pod \"konnectivity-agent-zs5vl\" (UID: \"0e086493-640f-43e4-a8bf-cc5f03d52abe\") " pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881371 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e086493-640f-43e4-a8bf-cc5f03d52abe-konnectivity-ca\") pod \"konnectivity-agent-zs5vl\" (UID: \"0e086493-640f-43e4-a8bf-cc5f03d52abe\") " pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881400 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-var-lib-kubelet\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881429 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-run-multus-certs\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.881465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881460 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-systemd-units\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881486 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-run-ovn-kubernetes\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881529 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f72393fd-77d3-4b0d-85c6-310f88907af1-ovn-node-metrics-cert\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881561 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-sysctl-d\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881591 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cb30e43-e55f-4dec-af3f-39960a5946cb-tmp-dir\") pod \"node-resolver-mtklw\" (UID: \"0cb30e43-e55f-4dec-af3f-39960a5946cb\") " pod="openshift-dns/node-resolver-mtklw" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881607 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880-serviceca\") pod \"node-ca-jkm76\" (UID: \"b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880\") " pod="openshift-image-registry/node-ca-jkm76" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881619 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-socket-dir-parent\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881664 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-run-netns\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.879217 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-kubelet\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881702 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-log-socket\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.880358 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-os-release\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881722 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-socket-dir-parent\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881788 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f72393fd-77d3-4b0d-85c6-310f88907af1-ovnkube-config\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881816 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-var-lib-kubelet\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881824 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-etc-kubernetes\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881872 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqjqr\" (UniqueName: \"kubernetes.io/projected/3283c783-546e-4a74-8c2c-40ab4dd734e6-kube-api-access-zqjqr\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881897 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-etc-kubernetes\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881926 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-run-netns\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.882150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.881985 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-cni-netd\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882255 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f72393fd-77d3-4b0d-85c6-310f88907af1-ovnkube-script-lib\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882294 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24b769d5-b98a-45d0-97fa-e67a68008449-dbus\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882328 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-cni-dir\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882084 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-cni-netd\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:27.882100 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882140 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-cnibin\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882181 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0cb30e43-e55f-4dec-af3f-39960a5946cb-hosts-file\") pod \"node-resolver-mtklw\" (UID: \"0cb30e43-e55f-4dec-af3f-39960a5946cb\") " pod="openshift-dns/node-resolver-mtklw" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882212 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-sysconfig\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882484 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-run-netns\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882039 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-log-socket\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:27.882556 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret podName:24b769d5-b98a-45d0-97fa-e67a68008449 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:28.382528658 +0000 UTC m=+3.061793641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret") pod "global-pull-secret-syncer-82c65" (UID: "24b769d5-b98a-45d0-97fa-e67a68008449") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882711 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-run-ovn-kubernetes\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882713 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-run\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.883046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882879 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-sysctl-d\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.883744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.883228 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cb30e43-e55f-4dec-af3f-39960a5946cb-tmp-dir\") pod \"node-resolver-mtklw\" (UID: \"0cb30e43-e55f-4dec-af3f-39960a5946cb\") " pod="openshift-dns/node-resolver-mtklw" Jul 16 10:32:27.883744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.883263 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f72393fd-77d3-4b0d-85c6-310f88907af1-ovnkube-script-lib\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.883744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.883298 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e086493-640f-43e4-a8bf-cc5f03d52abe-konnectivity-ca\") pod \"konnectivity-agent-zs5vl\" (UID: \"0e086493-640f-43e4-a8bf-cc5f03d52abe\") " pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:27.883744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.883327 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f72393fd-77d3-4b0d-85c6-310f88907af1-ovnkube-config\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.883744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.883346 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-var-lib-cni-multus\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.883744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.883380 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-systemd-units\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.883744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882080 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-system-cni-dir\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.883744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.883446 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-conf-dir\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.883744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.883576 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24b769d5-b98a-45d0-97fa-e67a68008449-dbus\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:27.883744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.883648 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-run-openvswitch\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.883744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.882083 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f72393fd-77d3-4b0d-85c6-310f88907af1-env-overrides\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.884355 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.883754 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-cni-dir\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.884355 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.884035 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3283c783-546e-4a74-8c2c-40ab4dd734e6-multus-daemon-config\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.884355 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.884045 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fae8c27e-c7eb-4f51-af72-4f0a88996042-etc-tuned\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.884355 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.884112 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-cni-bin\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.884355 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.884180 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-var-lib-cni-bin\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.884618 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.884503 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fda8e68-706b-4c7d-998d-ba89897e6457-os-release\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.884618 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.884508 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f72393fd-77d3-4b0d-85c6-310f88907af1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.884724 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.884705 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3283c783-546e-4a74-8c2c-40ab4dd734e6-cni-binary-copy\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.884802 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.884737 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24b769d5-b98a-45d0-97fa-e67a68008449-kubelet-config\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:27.884867 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:27.884831 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:27.884924 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:27.884906 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs podName:a6cb6750-2018-4424-b782-201c673ee132 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:28.384887638 +0000 UTC m=+3.064152815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs") pod "network-metrics-daemon-rdp5f" (UID: "a6cb6750-2018-4424-b782-201c673ee132") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:27.885079 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.885016 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3283c783-546e-4a74-8c2c-40ab4dd734e6-host-run-multus-certs\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.885079 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.885021 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fda8e68-706b-4c7d-998d-ba89897e6457-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.885235 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.885128 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fda8e68-706b-4c7d-998d-ba89897e6457-cni-binary-copy\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.885812 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.885748 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f72393fd-77d3-4b0d-85c6-310f88907af1-ovn-node-metrics-cert\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.885987 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.885934 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e086493-640f-43e4-a8bf-cc5f03d52abe-agent-certs\") pod \"konnectivity-agent-zs5vl\" (UID: \"0e086493-640f-43e4-a8bf-cc5f03d52abe\") " pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:27.886109 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.886082 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fda8e68-706b-4c7d-998d-ba89897e6457-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.886598 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.886580 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fae8c27e-c7eb-4f51-af72-4f0a88996042-tmp\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.896351 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:27.896328 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:27.896448 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:27.896354 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:27.896448 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:27.896369 2581 projected.go:196] Error preparing data for projected volume kube-api-access-9vwqw for pod openshift-network-diagnostics/network-check-target-lnndv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:27.896448 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:27.896440 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw podName:a79fe0b6-6a92-4396-b856-d50f5c5383a3 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:28.396422752 +0000 UTC m=+3.075687725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9vwqw" (UniqueName: "kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw") pod "network-check-target-lnndv" (UID: "a79fe0b6-6a92-4396-b856-d50f5c5383a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:27.896985 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.896927 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x68nl\" (UniqueName: \"kubernetes.io/projected/b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880-kube-api-access-x68nl\") pod \"node-ca-jkm76\" (UID: \"b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880\") " pod="openshift-image-registry/node-ca-jkm76" Jul 16 10:32:27.896985 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.896938 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqjqr\" (UniqueName: \"kubernetes.io/projected/3283c783-546e-4a74-8c2c-40ab4dd734e6-kube-api-access-zqjqr\") pod \"multus-ztj49\" (UID: \"3283c783-546e-4a74-8c2c-40ab4dd734e6\") " pod="openshift-multus/multus-ztj49" Jul 16 10:32:27.898745 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.898701 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l47zt\" (UniqueName: \"kubernetes.io/projected/fae8c27e-c7eb-4f51-af72-4f0a88996042-kube-api-access-l47zt\") pod \"tuned-p68l8\" (UID: \"fae8c27e-c7eb-4f51-af72-4f0a88996042\") " pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:27.899040 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.899018 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5sp8\" (UniqueName: \"kubernetes.io/projected/0fda8e68-706b-4c7d-998d-ba89897e6457-kube-api-access-p5sp8\") pod \"multus-additional-cni-plugins-7b4t8\" (UID: \"0fda8e68-706b-4c7d-998d-ba89897e6457\") " pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:27.899501 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.899479 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk7ng\" (UniqueName: \"kubernetes.io/projected/0cb30e43-e55f-4dec-af3f-39960a5946cb-kube-api-access-fk7ng\") pod \"node-resolver-mtklw\" (UID: \"0cb30e43-e55f-4dec-af3f-39960a5946cb\") " pod="openshift-dns/node-resolver-mtklw" Jul 16 10:32:27.900057 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.899998 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6v47\" (UniqueName: \"kubernetes.io/projected/f72393fd-77d3-4b0d-85c6-310f88907af1-kube-api-access-r6v47\") pod \"ovnkube-node-4n6zq\" (UID: \"f72393fd-77d3-4b0d-85c6-310f88907af1\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:27.900435 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.900411 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ljdp\" (UniqueName: \"kubernetes.io/projected/a6cb6750-2018-4424-b782-201c673ee132-kube-api-access-2ljdp\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:27.976950 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.976921 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" Jul 16 10:32:27.987759 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.987736 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jqlcl" Jul 16 10:32:27.995857 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:27.995833 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7b4t8" Jul 16 10:32:28.001735 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.001714 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ztj49" Jul 16 10:32:28.008845 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.008823 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:28.015854 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.015830 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jkm76" Jul 16 10:32:28.022572 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.022551 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:28.030483 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.030465 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p68l8" Jul 16 10:32:28.036192 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.036171 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mtklw" Jul 16 10:32:28.090225 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.090200 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 10:32:28.386672 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.386591 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:28.386672 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.386653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:28.386909 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:28.386774 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:28.386909 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:28.386848 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret podName:24b769d5-b98a-45d0-97fa-e67a68008449 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:29.386828912 +0000 UTC m=+4.066093871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret") pod "global-pull-secret-syncer-82c65" (UID: "24b769d5-b98a-45d0-97fa-e67a68008449") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:28.386992 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:28.386907 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:28.386992 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:28.386964 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs podName:a6cb6750-2018-4424-b782-201c673ee132 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:29.386951172 +0000 UTC m=+4.066216131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs") pod "network-metrics-daemon-rdp5f" (UID: "a6cb6750-2018-4424-b782-201c673ee132") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:28.487336 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.487293 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vwqw\" (UniqueName: \"kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw\") pod \"network-check-target-lnndv\" (UID: \"a79fe0b6-6a92-4396-b856-d50f5c5383a3\") " pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:28.487515 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:28.487498 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:28.487577 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:28.487529 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:28.487577 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:28.487544 2581 projected.go:196] Error preparing data for projected volume kube-api-access-9vwqw for pod openshift-network-diagnostics/network-check-target-lnndv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:28.487684 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:28.487617 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw podName:a79fe0b6-6a92-4396-b856-d50f5c5383a3 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:29.4875958 +0000 UTC m=+4.166860784 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vwqw" (UniqueName: "kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw") pod "network-check-target-lnndv" (UID: "a79fe0b6-6a92-4396-b856-d50f5c5383a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:28.550439 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:28.550397 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf72393fd_77d3_4b0d_85c6_310f88907af1.slice/crio-a2b7d4bd561652243b561c62f42cb7e08faeccc6907d8f1c082466f3414dc01f WatchSource:0}: Error finding container a2b7d4bd561652243b561c62f42cb7e08faeccc6907d8f1c082466f3414dc01f: Status 404 returned error can't find the container with id a2b7d4bd561652243b561c62f42cb7e08faeccc6907d8f1c082466f3414dc01f Jul 16 10:32:28.553397 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:28.553361 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19ca7547_d371_4c98_b81d_9b0812440419.slice/crio-1c54ac4c15b503ca66970254ce0c0074b633da60fbe666d06a3edc0af56a9ea1 WatchSource:0}: Error finding container 1c54ac4c15b503ca66970254ce0c0074b633da60fbe666d06a3edc0af56a9ea1: Status 404 returned error can't find the container with id 1c54ac4c15b503ca66970254ce0c0074b633da60fbe666d06a3edc0af56a9ea1 Jul 16 10:32:28.556960 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:28.556935 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e086493_640f_43e4_a8bf_cc5f03d52abe.slice/crio-7f6e6cbf13f734ef8f2ef4b6ced034a5d0c792b2d6f07565ec365e504a20c7d4 WatchSource:0}: Error finding container 7f6e6cbf13f734ef8f2ef4b6ced034a5d0c792b2d6f07565ec365e504a20c7d4: Status 404 returned error can't find the container with id 7f6e6cbf13f734ef8f2ef4b6ced034a5d0c792b2d6f07565ec365e504a20c7d4 Jul 16 10:32:28.559855 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:28.559795 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3283c783_546e_4a74_8c2c_40ab4dd734e6.slice/crio-98eb9ee054d322e159f10411c570000dd64be0899cf54203d8aec9cd867b41c9 WatchSource:0}: Error finding container 98eb9ee054d322e159f10411c570000dd64be0899cf54203d8aec9cd867b41c9: Status 404 returned error can't find the container with id 98eb9ee054d322e159f10411c570000dd64be0899cf54203d8aec9cd867b41c9 Jul 16 10:32:28.560986 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:28.560928 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb30e43_e55f_4dec_af3f_39960a5946cb.slice/crio-d7991dad865247ca92869e6ae818ccf6898584714411d367e547be7391e7cc26 WatchSource:0}: Error finding container d7991dad865247ca92869e6ae818ccf6898584714411d367e547be7391e7cc26: Status 404 returned error can't find the container with id d7991dad865247ca92869e6ae818ccf6898584714411d367e547be7391e7cc26 Jul 16 10:32:28.561873 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:28.561733 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fda8e68_706b_4c7d_998d_ba89897e6457.slice/crio-b8ab9c0e8b64aa2b2319cfeadaa60f7acf6d6ab353cbb157b9ebb082755b8cd9 WatchSource:0}: Error finding container b8ab9c0e8b64aa2b2319cfeadaa60f7acf6d6ab353cbb157b9ebb082755b8cd9: Status 404 returned error can't find the container with id b8ab9c0e8b64aa2b2319cfeadaa60f7acf6d6ab353cbb157b9ebb082755b8cd9 Jul 16 10:32:28.562743 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:28.562694 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeded4b36_847f_4ecc_9f18_4e57499337bf.slice/crio-12a47e1197641084176f613e10efd4cf9b7b1f8cae8798eb70e5354f1a65a80d WatchSource:0}: Error finding container 12a47e1197641084176f613e10efd4cf9b7b1f8cae8798eb70e5354f1a65a80d: Status 404 returned error can't find the container with id 12a47e1197641084176f613e10efd4cf9b7b1f8cae8798eb70e5354f1a65a80d Jul 16 10:32:28.564006 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:28.563983 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb115d8a4_0ad2_4dd2_9f96_7f05fb7b3880.slice/crio-8f9a3ddbc75f351dfe941a9952137d5ce380fa798bc61a3c8aaddb0fdf8d140e WatchSource:0}: Error finding container 8f9a3ddbc75f351dfe941a9952137d5ce380fa798bc61a3c8aaddb0fdf8d140e: Status 404 returned error can't find the container with id 8f9a3ddbc75f351dfe941a9952137d5ce380fa798bc61a3c8aaddb0fdf8d140e Jul 16 10:32:28.565641 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:32:28.565619 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae8c27e_c7eb_4f51_af72_4f0a88996042.slice/crio-758885b62dbcd0496e529a03656b6ec25820c9365210aea4a34770469f907520 WatchSource:0}: Error finding container 758885b62dbcd0496e529a03656b6ec25820c9365210aea4a34770469f907520: Status 404 returned error can't find the container with id 758885b62dbcd0496e529a03656b6ec25820c9365210aea4a34770469f907520 Jul 16 10:32:28.717466 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.717146 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 10:27:28 +0000 UTC" deadline="2027-12-25 09:28:34.808791029 +0000 UTC" Jul 16 10:32:28.717466 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.717406 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12646h56m6.09139342s" Jul 16 10:32:28.808938 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.808899 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jqlcl" event={"ID":"19ca7547-d371-4c98-b81d-9b0812440419","Type":"ContainerStarted","Data":"1c54ac4c15b503ca66970254ce0c0074b633da60fbe666d06a3edc0af56a9ea1"} Jul 16 10:32:28.810271 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.810233 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p68l8" event={"ID":"fae8c27e-c7eb-4f51-af72-4f0a88996042","Type":"ContainerStarted","Data":"758885b62dbcd0496e529a03656b6ec25820c9365210aea4a34770469f907520"} Jul 16 10:32:28.811587 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.811546 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jkm76" event={"ID":"b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880","Type":"ContainerStarted","Data":"8f9a3ddbc75f351dfe941a9952137d5ce380fa798bc61a3c8aaddb0fdf8d140e"} Jul 16 10:32:28.812814 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.812780 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" event={"ID":"f72393fd-77d3-4b0d-85c6-310f88907af1","Type":"ContainerStarted","Data":"a2b7d4bd561652243b561c62f42cb7e08faeccc6907d8f1c082466f3414dc01f"} Jul 16 10:32:28.814953 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.814894 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-37.ec2.internal" event={"ID":"26a59dee568ae6e70ff513d52676f317","Type":"ContainerStarted","Data":"b59e2b8d32cb84946c599ffdee752470fb11a9c0313cdc352551603a236d5de2"} Jul 16 10:32:28.816354 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.816327 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" event={"ID":"eded4b36-847f-4ecc-9f18-4e57499337bf","Type":"ContainerStarted","Data":"12a47e1197641084176f613e10efd4cf9b7b1f8cae8798eb70e5354f1a65a80d"} Jul 16 10:32:28.817650 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.817614 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b4t8" event={"ID":"0fda8e68-706b-4c7d-998d-ba89897e6457","Type":"ContainerStarted","Data":"b8ab9c0e8b64aa2b2319cfeadaa60f7acf6d6ab353cbb157b9ebb082755b8cd9"} Jul 16 10:32:28.818817 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.818786 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mtklw" event={"ID":"0cb30e43-e55f-4dec-af3f-39960a5946cb","Type":"ContainerStarted","Data":"d7991dad865247ca92869e6ae818ccf6898584714411d367e547be7391e7cc26"} Jul 16 10:32:28.820167 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.820125 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ztj49" event={"ID":"3283c783-546e-4a74-8c2c-40ab4dd734e6","Type":"ContainerStarted","Data":"98eb9ee054d322e159f10411c570000dd64be0899cf54203d8aec9cd867b41c9"} Jul 16 10:32:28.821409 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:28.821384 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zs5vl" event={"ID":"0e086493-640f-43e4-a8bf-cc5f03d52abe","Type":"ContainerStarted","Data":"7f6e6cbf13f734ef8f2ef4b6ced034a5d0c792b2d6f07565ec365e504a20c7d4"} Jul 16 10:32:29.395248 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:29.395209 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:29.395431 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:29.395304 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:29.395431 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:29.395421 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:29.395555 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:29.395485 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret podName:24b769d5-b98a-45d0-97fa-e67a68008449 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:31.395466082 +0000 UTC m=+6.074731042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret") pod "global-pull-secret-syncer-82c65" (UID: "24b769d5-b98a-45d0-97fa-e67a68008449") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:29.396064 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:29.396042 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:29.396190 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:29.396113 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs podName:a6cb6750-2018-4424-b782-201c673ee132 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:31.396084039 +0000 UTC m=+6.075348998 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs") pod "network-metrics-daemon-rdp5f" (UID: "a6cb6750-2018-4424-b782-201c673ee132") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:29.496229 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:29.496165 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vwqw\" (UniqueName: \"kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw\") pod \"network-check-target-lnndv\" (UID: \"a79fe0b6-6a92-4396-b856-d50f5c5383a3\") " pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:29.496420 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:29.496375 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:29.496420 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:29.496399 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:29.496420 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:29.496415 2581 projected.go:196] Error preparing data for projected volume kube-api-access-9vwqw for pod openshift-network-diagnostics/network-check-target-lnndv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:29.496582 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:29.496476 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw podName:a79fe0b6-6a92-4396-b856-d50f5c5383a3 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:31.496456838 +0000 UTC m=+6.175721819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vwqw" (UniqueName: "kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw") pod "network-check-target-lnndv" (UID: "a79fe0b6-6a92-4396-b856-d50f5c5383a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:29.718123 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:29.717961 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 10:27:28 +0000 UTC" deadline="2028-04-02 19:17:10.427865692 +0000 UTC" Jul 16 10:32:29.718123 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:29.718034 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15032h44m40.709836362s" Jul 16 10:32:29.802005 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:29.801211 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:29.802005 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:29.801371 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:29.802005 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:29.801857 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:29.802005 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:29.801959 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:29.802628 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:29.802482 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:29.802628 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:29.802575 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:31.411691 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:31.411300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:31.411691 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:31.411359 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:31.411691 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:31.411473 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:31.411691 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:31.411552 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:31.411691 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:31.411578 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret podName:24b769d5-b98a-45d0-97fa-e67a68008449 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:35.411556161 +0000 UTC m=+10.090821146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret") pod "global-pull-secret-syncer-82c65" (UID: "24b769d5-b98a-45d0-97fa-e67a68008449") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:31.411691 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:31.411609 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs podName:a6cb6750-2018-4424-b782-201c673ee132 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:35.411591944 +0000 UTC m=+10.090856904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs") pod "network-metrics-daemon-rdp5f" (UID: "a6cb6750-2018-4424-b782-201c673ee132") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:31.512402 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:31.512361 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vwqw\" (UniqueName: \"kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw\") pod \"network-check-target-lnndv\" (UID: \"a79fe0b6-6a92-4396-b856-d50f5c5383a3\") " pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:31.512579 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:31.512564 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:31.512637 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:31.512588 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:31.512637 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:31.512602 2581 projected.go:196] Error preparing data for projected volume kube-api-access-9vwqw for pod openshift-network-diagnostics/network-check-target-lnndv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:31.512747 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:31.512664 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw podName:a79fe0b6-6a92-4396-b856-d50f5c5383a3 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:35.512646238 +0000 UTC m=+10.191911221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vwqw" (UniqueName: "kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw") pod "network-check-target-lnndv" (UID: "a79fe0b6-6a92-4396-b856-d50f5c5383a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:31.800953 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:31.800872 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:31.801131 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:31.801034 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:31.801452 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:31.801428 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:31.801595 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:31.801564 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:31.801665 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:31.801631 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:31.801730 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:31.801712 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:33.801607 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:33.801572 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:33.801607 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:33.801574 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:33.802157 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:33.801755 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:33.802157 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:33.801781 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:33.802157 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:33.801927 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:33.802157 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:33.802011 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:35.448251 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:35.448147 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:35.448251 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:35.448206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:35.448802 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:35.448325 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:35.448802 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:35.448345 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:35.448802 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:35.448412 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret podName:24b769d5-b98a-45d0-97fa-e67a68008449 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:43.448389015 +0000 UTC m=+18.127653989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret") pod "global-pull-secret-syncer-82c65" (UID: "24b769d5-b98a-45d0-97fa-e67a68008449") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:35.448802 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:35.448433 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs podName:a6cb6750-2018-4424-b782-201c673ee132 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:43.448422444 +0000 UTC m=+18.127687416 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs") pod "network-metrics-daemon-rdp5f" (UID: "a6cb6750-2018-4424-b782-201c673ee132") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:35.548850 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:35.548811 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vwqw\" (UniqueName: \"kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw\") pod \"network-check-target-lnndv\" (UID: \"a79fe0b6-6a92-4396-b856-d50f5c5383a3\") " pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:35.549056 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:35.549034 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:35.549056 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:35.549054 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:35.549181 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:35.549067 2581 projected.go:196] Error preparing data for projected volume kube-api-access-9vwqw for pod openshift-network-diagnostics/network-check-target-lnndv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:35.549181 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:35.549131 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw podName:a79fe0b6-6a92-4396-b856-d50f5c5383a3 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:43.549112604 +0000 UTC m=+18.228377564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vwqw" (UniqueName: "kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw") pod "network-check-target-lnndv" (UID: "a79fe0b6-6a92-4396-b856-d50f5c5383a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:35.801115 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:35.800974 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:35.801115 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:35.801070 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:35.801325 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:35.801114 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:35.801325 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:35.801182 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:35.801325 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:35.801284 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:35.801505 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:35.801349 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:37.800188 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:37.800031 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:37.800188 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:37.800167 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:37.800188 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:37.800031 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:37.800758 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:37.800253 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:37.800758 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:37.800028 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:37.800758 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:37.800341 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:39.800678 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:39.800645 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:39.800678 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:39.800675 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:39.801090 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:39.800645 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:39.801090 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:39.800777 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:39.801090 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:39.800877 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:39.801090 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:39.800963 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:41.800007 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:41.799971 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:41.800435 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:41.799983 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:41.800435 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:41.800105 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:41.800435 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:41.800157 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:41.800435 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:41.799980 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:41.800435 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:41.800276 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:43.508334 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:43.508302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:43.508710 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:43.508348 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:43.508710 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:43.508434 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:43.508710 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:43.508435 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:43.508710 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:43.508486 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs podName:a6cb6750-2018-4424-b782-201c673ee132 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.508472496 +0000 UTC m=+34.187737466 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs") pod "network-metrics-daemon-rdp5f" (UID: "a6cb6750-2018-4424-b782-201c673ee132") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:43.508710 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:43.508500 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret podName:24b769d5-b98a-45d0-97fa-e67a68008449 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.508492361 +0000 UTC m=+34.187757331 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret") pod "global-pull-secret-syncer-82c65" (UID: "24b769d5-b98a-45d0-97fa-e67a68008449") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:43.609172 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:43.609139 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vwqw\" (UniqueName: \"kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw\") pod \"network-check-target-lnndv\" (UID: \"a79fe0b6-6a92-4396-b856-d50f5c5383a3\") " pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:43.609347 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:43.609327 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:43.609415 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:43.609350 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:43.609415 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:43.609365 2581 projected.go:196] Error preparing data for projected volume kube-api-access-9vwqw for pod openshift-network-diagnostics/network-check-target-lnndv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:43.609522 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:43.609459 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw podName:a79fe0b6-6a92-4396-b856-d50f5c5383a3 nodeName:}" failed. No retries permitted until 2026-07-16 10:32:59.609420894 +0000 UTC m=+34.288685874 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vwqw" (UniqueName: "kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw") pod "network-check-target-lnndv" (UID: "a79fe0b6-6a92-4396-b856-d50f5c5383a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:43.800644 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:43.800549 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:43.800821 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:43.800555 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:43.800821 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:43.800684 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:43.800821 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:43.800711 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:43.800821 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:43.800806 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:43.801012 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:43.800894 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:45.800785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:45.800752 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:45.800785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:45.800767 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:45.801271 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:45.800882 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:45.801271 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:45.800897 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:45.801271 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:45.801011 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:45.801271 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:45.801103 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:47.801122 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.800826 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:47.801807 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.800826 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:47.801807 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:47.801211 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:47.801807 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.800863 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:47.801807 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:47.801300 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:47.801807 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:47.801346 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:47.878531 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.877874 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zs5vl" event={"ID":"0e086493-640f-43e4-a8bf-cc5f03d52abe","Type":"ContainerStarted","Data":"59d2d79e663375bb42714c72457eaafafc8637e479c87dfc271bcbdb61b3798d"} Jul 16 10:32:47.881020 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.880981 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p68l8" event={"ID":"fae8c27e-c7eb-4f51-af72-4f0a88996042","Type":"ContainerStarted","Data":"6d53a9af6936e82ab38fd43e257990d16a399efab79e0ee5fcc51df97e927c07"} Jul 16 10:32:47.882617 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.882585 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jkm76" event={"ID":"b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880","Type":"ContainerStarted","Data":"335d2636b86887cafa09cf75f325d785e08283f6ab25c08e6d5c6ed15daefdb1"} Jul 16 10:32:47.885667 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.885641 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:32:47.886076 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.886050 2581 generic.go:356] "Generic (PLEG): container finished" podID="f72393fd-77d3-4b0d-85c6-310f88907af1" containerID="bed8da01cb1b8aa8a44ef0c00f8d9d7f5e4ccab1a48c2a86567ac3ce897de694" exitCode=1 Jul 16 10:32:47.886166 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.886112 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" event={"ID":"f72393fd-77d3-4b0d-85c6-310f88907af1","Type":"ContainerStarted","Data":"d362838e69a8859ed990c2f5fad67047636d70c90d8b5aaf0f6e3d2d367f088a"} Jul 16 10:32:47.886222 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.886186 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" event={"ID":"f72393fd-77d3-4b0d-85c6-310f88907af1","Type":"ContainerStarted","Data":"542359a0c1e43e7d833b0b9ad85cd5e85018926f79e3b6a6da9ec37c6006c443"} Jul 16 10:32:47.886263 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.886236 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" event={"ID":"f72393fd-77d3-4b0d-85c6-310f88907af1","Type":"ContainerStarted","Data":"924c25289329f3a57beda1afcedf52974ba0c9fd4c3394045a84f0d1b6402688"} Jul 16 10:32:47.886304 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.886285 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" event={"ID":"f72393fd-77d3-4b0d-85c6-310f88907af1","Type":"ContainerStarted","Data":"cd9b4b019455841bbb023a10d43843c957e9a02ba825a8ac45a5d2e16b67ad55"} Jul 16 10:32:47.886347 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.886330 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" event={"ID":"f72393fd-77d3-4b0d-85c6-310f88907af1","Type":"ContainerDied","Data":"bed8da01cb1b8aa8a44ef0c00f8d9d7f5e4ccab1a48c2a86567ac3ce897de694"} Jul 16 10:32:47.886383 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.886363 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" event={"ID":"f72393fd-77d3-4b0d-85c6-310f88907af1","Type":"ContainerStarted","Data":"98e113e4da168dbfeca57dd434e21e07d54d1fc96bb387f164e3a2becf9a0f97"} Jul 16 10:32:47.887869 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.887843 2581 generic.go:356] "Generic (PLEG): container finished" podID="446caf796df52d1b266f588acb053144" containerID="20582c1096fbff8eca605ab3e08b738a4f380ee6cf4c8a0a5025cbe34c18c57a" exitCode=0 Jul 16 10:32:47.887982 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.887930 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" event={"ID":"446caf796df52d1b266f588acb053144","Type":"ContainerDied","Data":"20582c1096fbff8eca605ab3e08b738a4f380ee6cf4c8a0a5025cbe34c18c57a"} Jul 16 10:32:47.889841 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.889814 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" event={"ID":"eded4b36-847f-4ecc-9f18-4e57499337bf","Type":"ContainerStarted","Data":"ceb0cde180f20cdeb621dbb2849c07d0c21d9b2efe00c07aa49e15ace1254791"} Jul 16 10:32:47.891567 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.891543 2581 generic.go:356] "Generic (PLEG): container finished" podID="0fda8e68-706b-4c7d-998d-ba89897e6457" containerID="e8f57f895a950ec82d2ffbf387a310f155e85e1b9c03714baada8299da801fa8" exitCode=0 Jul 16 10:32:47.891709 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.891608 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b4t8" event={"ID":"0fda8e68-706b-4c7d-998d-ba89897e6457","Type":"ContainerDied","Data":"e8f57f895a950ec82d2ffbf387a310f155e85e1b9c03714baada8299da801fa8"} Jul 16 10:32:47.893570 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.893552 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mtklw" event={"ID":"0cb30e43-e55f-4dec-af3f-39960a5946cb","Type":"ContainerStarted","Data":"44c26d8fc813655020362834bb72a22ce4a4e16e9f7c2e73eb14bbb3ed804084"} Jul 16 10:32:47.895272 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.895248 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ztj49" event={"ID":"3283c783-546e-4a74-8c2c-40ab4dd734e6","Type":"ContainerStarted","Data":"f4e41d3d7d16cb634c8ff46cd9a706c86db44606bde32ff824f1005b7efba287"} Jul 16 10:32:47.896619 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.896561 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zs5vl" podStartSLOduration=7.079451774 podStartE2EDuration="21.896549665s" totalImagesPullingTime="14.817097891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.896302718 +0000 UTC m=+22.575567700" watchObservedRunningTime="2026-07-16 10:32:47.896549665 +0000 UTC m=+22.575814645" Jul 16 10:32:47.896740 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.896651 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-37.ec2.internal" podStartSLOduration=20.896644186 podStartE2EDuration="20.896644186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:28.835390549 +0000 UTC m=+3.514655527" watchObservedRunningTime="2026-07-16 10:32:47.896644186 +0000 UTC m=+22.575909169" Jul 16 10:32:47.922558 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.922421 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ztj49" podStartSLOduration=3.240604206 podStartE2EDuration="21.922400097s" totalImagesPullingTime="18.681795891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.921945863 +0000 UTC m=+22.601210844" watchObservedRunningTime="2026-07-16 10:32:47.922400097 +0000 UTC m=+22.601665081" Jul 16 10:32:47.948006 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.947920 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jkm76" podStartSLOduration=3.3350959700000002 podStartE2EDuration="21.947900064s" totalImagesPullingTime="18.612804094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.947071303 +0000 UTC m=+22.626336286" watchObservedRunningTime="2026-07-16 10:32:47.947900064 +0000 UTC m=+22.627165045" Jul 16 10:32:47.987943 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.987891 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mtklw" podStartSLOduration=3.362251023 podStartE2EDuration="21.987869721s" totalImagesPullingTime="18.625618698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.987729256 +0000 UTC m=+22.666994239" watchObservedRunningTime="2026-07-16 10:32:47.987869721 +0000 UTC m=+22.667134703" Jul 16 10:32:47.988324 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:47.988283 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p68l8" podStartSLOduration=3.344202884 podStartE2EDuration="21.98827278s" totalImagesPullingTime="18.644069896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:47.968046644 +0000 UTC m=+22.647311629" watchObservedRunningTime="2026-07-16 10:32:47.98827278 +0000 UTC m=+22.667537762" Jul 16 10:32:48.628933 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:48.628911 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 10:32:48.737051 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:48.736901 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T10:32:48.628930759Z","UUID":"e57e45a0-eeb3-4b66-8bbd-dc29cee20c3d","Handler":null,"Name":"","Endpoint":""} Jul 16 10:32:48.738578 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:48.738557 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 10:32:48.738707 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:48.738598 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 10:32:48.899866 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:48.899823 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" event={"ID":"446caf796df52d1b266f588acb053144","Type":"ContainerStarted","Data":"f260684452d9565dd877f614996de3666a83d42c9ee45209baff0e852743c12f"} Jul 16 10:32:48.901912 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:48.901881 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" event={"ID":"eded4b36-847f-4ecc-9f18-4e57499337bf","Type":"ContainerStarted","Data":"6bd920b46216d8006acc975a54fba9f6bdf96e4730d28f31d7074132cf267c30"} Jul 16 10:32:48.903486 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:48.903451 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jqlcl" event={"ID":"19ca7547-d371-4c98-b81d-9b0812440419","Type":"ContainerStarted","Data":"62151e0a3dd34df42dfe2718c9fd49845346cb0aedadbb877edde719c3876985"} Jul 16 10:32:48.917813 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:48.917760 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-37.ec2.internal" podStartSLOduration=21.917744795 podStartE2EDuration="21.917744795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:48.917480449 +0000 UTC m=+23.596745429" watchObservedRunningTime="2026-07-16 10:32:48.917744795 +0000 UTC m=+23.597009776" Jul 16 10:32:48.941441 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:48.941367 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jqlcl" podStartSLOduration=4.309228551 podStartE2EDuration="22.941351894s" totalImagesPullingTime="18.632123343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:48.941024449 +0000 UTC m=+23.620289428" watchObservedRunningTime="2026-07-16 10:32:48.941351894 +0000 UTC m=+23.620616874" Jul 16 10:32:49.524210 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:49.524179 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:49.800804 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:49.800726 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:49.800804 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:49.800771 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:49.801068 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:49.800725 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:49.801068 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:49.800855 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:49.801068 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:49.800977 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:49.801201 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:49.801063 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:49.911477 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:49.911423 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" event={"ID":"eded4b36-847f-4ecc-9f18-4e57499337bf","Type":"ContainerStarted","Data":"9adf9a8f8b6e1b2a9fba6b448ee0770d0a47b26d1cd8a233955bc7323e82f8d1"} Jul 16 10:32:49.936376 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:49.936320 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h5p9s" podStartSLOduration=7.19405147 podStartE2EDuration="23.936304517s" totalImagesPullingTime="16.742253047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:49.936206166 +0000 UTC m=+24.615471165" watchObservedRunningTime="2026-07-16 10:32:49.936304517 +0000 UTC m=+24.615569498" Jul 16 10:32:50.917563 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:50.917535 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:32:50.918094 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:50.917949 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" event={"ID":"f72393fd-77d3-4b0d-85c6-310f88907af1","Type":"ContainerStarted","Data":"f16121a3e33a86dc50f26c1c6c35d2728fef0094c65dbacc55e199368f4ae81c"} Jul 16 10:32:51.800984 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:51.800755 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:51.801215 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:51.800871 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:51.801215 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:51.801083 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:51.801215 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:51.800899 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:51.801215 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:51.801155 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:51.801459 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:51.801279 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:52.052468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:52.052386 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:52.053058 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:52.053042 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:52.925319 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:52.925148 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:32:52.925728 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:52.925703 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" event={"ID":"f72393fd-77d3-4b0d-85c6-310f88907af1","Type":"ContainerStarted","Data":"575d68d3c8e5af6707909bb5c2b6532dbd8490a4cec69d6bb91f518a0da6c631"} Jul 16 10:32:52.925940 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:52.925914 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:52.926126 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:52.926110 2581 scope.go:117] "RemoveContainer" containerID="bed8da01cb1b8aa8a44ef0c00f8d9d7f5e4ccab1a48c2a86567ac3ce897de694" Jul 16 10:32:52.927628 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:52.927610 2581 generic.go:356] "Generic (PLEG): container finished" podID="0fda8e68-706b-4c7d-998d-ba89897e6457" containerID="fd3ddcc454ad9b7085fe5a3cba9982f0ac31b71809a5ea94be1602919270295b" exitCode=0 Jul 16 10:32:52.927738 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:52.927711 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b4t8" event={"ID":"0fda8e68-706b-4c7d-998d-ba89897e6457","Type":"ContainerDied","Data":"fd3ddcc454ad9b7085fe5a3cba9982f0ac31b71809a5ea94be1602919270295b"} Jul 16 10:32:52.928555 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:52.928481 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zs5vl" Jul 16 10:32:52.944592 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:52.942054 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:53.799889 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:53.799854 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:53.800707 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:53.799854 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:53.800707 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:53.799966 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:53.800707 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:53.799854 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:53.800707 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:53.800053 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:53.800707 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:53.800119 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:53.934453 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:53.934386 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:32:53.934830 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:53.934803 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" event={"ID":"f72393fd-77d3-4b0d-85c6-310f88907af1","Type":"ContainerStarted","Data":"1b9ebc2a23de7ced1928ce3d50157677a22a410cd16665c118f3d1d0f6cd8a09"} Jul 16 10:32:53.934966 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:53.934949 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:53.935032 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:53.935021 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:53.949571 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:53.949546 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:32:53.979210 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:53.979165 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" podStartSLOduration=9.288872586 podStartE2EDuration="27.979148844s" totalImagesPullingTime="18.690276258s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:32:53.978858643 +0000 UTC m=+28.658123624" watchObservedRunningTime="2026-07-16 10:32:53.979148844 +0000 UTC m=+28.658413863" Jul 16 10:32:54.542975 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:54.542900 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lnndv"] Jul 16 10:32:54.543129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:54.543084 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:54.543223 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:54.543200 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:54.546958 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:54.546935 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-82c65"] Jul 16 10:32:54.547106 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:54.547028 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:54.547160 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:54.547130 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:54.547740 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:54.547717 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rdp5f"] Jul 16 10:32:54.547846 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:54.547832 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:54.547953 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:54.547928 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:54.938640 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:54.938607 2581 generic.go:356] "Generic (PLEG): container finished" podID="0fda8e68-706b-4c7d-998d-ba89897e6457" containerID="db38d4071ce1d6ad751015668de3c5547b247e2f51f527b4aee78de9a6e430c5" exitCode=0 Jul 16 10:32:54.939002 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:54.938707 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b4t8" event={"ID":"0fda8e68-706b-4c7d-998d-ba89897e6457","Type":"ContainerDied","Data":"db38d4071ce1d6ad751015668de3c5547b247e2f51f527b4aee78de9a6e430c5"} Jul 16 10:32:55.801052 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:55.800864 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:55.801238 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:55.800951 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:55.801238 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:55.801111 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:55.801238 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:55.801201 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:56.800355 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:56.800275 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:56.800983 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:56.800414 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:56.946192 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:56.946157 2581 generic.go:356] "Generic (PLEG): container finished" podID="0fda8e68-706b-4c7d-998d-ba89897e6457" containerID="3a98e5dbecd323e48a662b81a800cbf6aa552d8b44f1b971bd5f64f33a4f85a3" exitCode=0 Jul 16 10:32:56.946370 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:56.946218 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b4t8" event={"ID":"0fda8e68-706b-4c7d-998d-ba89897e6457","Type":"ContainerDied","Data":"3a98e5dbecd323e48a662b81a800cbf6aa552d8b44f1b971bd5f64f33a4f85a3"} Jul 16 10:32:57.800183 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:57.800147 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:57.800364 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:57.800274 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:32:57.800772 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:57.800147 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:57.800834 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:57.800762 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:58.799918 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:58.799883 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:58.800094 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:58.800012 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rdp5f" podUID="a6cb6750-2018-4424-b782-201c673ee132" Jul 16 10:32:59.530680 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:59.530641 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:59.531336 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:59.530714 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:32:59.531336 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:59.530841 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:59.531336 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:59.530860 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:59.531336 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:59.530925 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs podName:a6cb6750-2018-4424-b782-201c673ee132 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:31.53090231 +0000 UTC m=+66.210167283 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs") pod "network-metrics-daemon-rdp5f" (UID: "a6cb6750-2018-4424-b782-201c673ee132") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 10:32:59.531336 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:59.530951 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret podName:24b769d5-b98a-45d0-97fa-e67a68008449 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:31.530940616 +0000 UTC m=+66.210205574 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret") pod "global-pull-secret-syncer-82c65" (UID: "24b769d5-b98a-45d0-97fa-e67a68008449") : object "kube-system"/"original-pull-secret" not registered Jul 16 10:32:59.631279 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:59.631241 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vwqw\" (UniqueName: \"kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw\") pod \"network-check-target-lnndv\" (UID: \"a79fe0b6-6a92-4396-b856-d50f5c5383a3\") " pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:59.631468 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:59.631444 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 10:32:59.631530 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:59.631468 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 10:32:59.631530 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:59.631482 2581 projected.go:196] Error preparing data for projected volume kube-api-access-9vwqw for pod openshift-network-diagnostics/network-check-target-lnndv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:59.631616 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:59.631536 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw podName:a79fe0b6-6a92-4396-b856-d50f5c5383a3 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:31.631523077 +0000 UTC m=+66.310788052 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vwqw" (UniqueName: "kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw") pod "network-check-target-lnndv" (UID: "a79fe0b6-6a92-4396-b856-d50f5c5383a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 10:32:59.800173 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:59.800092 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:32:59.800327 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:59.800224 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-82c65" podUID="24b769d5-b98a-45d0-97fa-e67a68008449" Jul 16 10:32:59.800327 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:32:59.800298 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:32:59.800426 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:32:59.800386 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lnndv" podUID="a79fe0b6-6a92-4396-b856-d50f5c5383a3" Jul 16 10:33:00.198038 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.198008 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-37.ec2.internal" event="NodeReady" Jul 16 10:33:00.198218 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.198142 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 10:33:00.238039 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.238006 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-788b9dd6d-fnvz6"] Jul 16 10:33:00.240525 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.240502 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.243513 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.243478 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 10:33:00.243513 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.243511 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 10:33:00.243708 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.243539 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9j5q6\"" Jul 16 10:33:00.243708 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.243489 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 10:33:00.248912 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.248823 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 10:33:00.255549 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.254960 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n8m4p"] Jul 16 10:33:00.257407 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.257388 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9gnph"] Jul 16 10:33:00.257505 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.257486 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.259124 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.259103 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9gnph" Jul 16 10:33:00.260185 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.260164 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 10:33:00.260185 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.260179 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x6pbd\"" Jul 16 10:33:00.260346 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.260202 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 10:33:00.261744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.261729 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-274x8\"" Jul 16 10:33:00.261887 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.261755 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 10:33:00.261991 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.261816 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 10:33:00.262588 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.262565 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 10:33:00.267107 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.267086 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-788b9dd6d-fnvz6"] Jul 16 10:33:00.270977 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.270951 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9gnph"] Jul 16 10:33:00.271076 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.270998 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n8m4p"] Jul 16 10:33:00.336905 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.336863 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d8sm\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-kube-api-access-7d8sm\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.337070 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.336939 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-registry-certificates\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.337070 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.336987 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.337070 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.337018 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d6b706-1a35-49b4-ae0d-a297613a7546-config-volume\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.337070 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.337057 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-installation-pull-secrets\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.337297 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.337106 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffj6p\" (UniqueName: \"kubernetes.io/projected/a4d6b706-1a35-49b4-ae0d-a297613a7546-kube-api-access-ffj6p\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.337297 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.337145 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51dbb08d-7943-4048-90ac-422e0d32f177-ca-trust-extracted\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.337297 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.337196 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4n6p\" (UniqueName: \"kubernetes.io/projected/7e39891c-14f3-4ac9-948d-dcc8ded4165f-kube-api-access-t4n6p\") pod \"ingress-canary-9gnph\" (UID: \"7e39891c-14f3-4ac9-948d-dcc8ded4165f\") " pod="openshift-ingress-canary/ingress-canary-9gnph" Jul 16 10:33:00.337297 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.337267 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.337531 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.337298 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-trusted-ca\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.337531 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.337321 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-bound-sa-token\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.337531 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.337361 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-image-registry-private-configuration\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.337531 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.337419 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4d6b706-1a35-49b4-ae0d-a297613a7546-tmp-dir\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.337531 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.337454 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e39891c-14f3-4ac9-948d-dcc8ded4165f-cert\") pod \"ingress-canary-9gnph\" (UID: \"7e39891c-14f3-4ac9-948d-dcc8ded4165f\") " pod="openshift-ingress-canary/ingress-canary-9gnph" Jul 16 10:33:00.438242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438199 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.438411 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438253 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d6b706-1a35-49b4-ae0d-a297613a7546-config-volume\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.438411 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438282 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-installation-pull-secrets\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.438411 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438316 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffj6p\" (UniqueName: \"kubernetes.io/projected/a4d6b706-1a35-49b4-ae0d-a297613a7546-kube-api-access-ffj6p\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.438411 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438342 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51dbb08d-7943-4048-90ac-422e0d32f177-ca-trust-extracted\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.438411 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438367 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4n6p\" (UniqueName: \"kubernetes.io/projected/7e39891c-14f3-4ac9-948d-dcc8ded4165f-kube-api-access-t4n6p\") pod \"ingress-canary-9gnph\" (UID: \"7e39891c-14f3-4ac9-948d-dcc8ded4165f\") " pod="openshift-ingress-canary/ingress-canary-9gnph" Jul 16 10:33:00.438411 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:00.438380 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:33:00.438411 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:00.438402 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788b9dd6d-fnvz6: secret "image-registry-tls" not found Jul 16 10:33:00.438809 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:00.438465 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls podName:51dbb08d-7943-4048-90ac-422e0d32f177 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:00.938443503 +0000 UTC m=+35.617708481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls") pod "image-registry-788b9dd6d-fnvz6" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177") : secret "image-registry-tls" not found Jul 16 10:33:00.438809 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438578 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.438809 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438635 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-trusted-ca\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.438809 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:00.438653 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:33:00.438809 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-bound-sa-token\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.438809 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:00.438717 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls podName:a4d6b706-1a35-49b4-ae0d-a297613a7546 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:00.938702357 +0000 UTC m=+35.617967330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls") pod "dns-default-n8m4p" (UID: "a4d6b706-1a35-49b4-ae0d-a297613a7546") : secret "dns-default-metrics-tls" not found Jul 16 10:33:00.438809 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438751 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-image-registry-private-configuration\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.438809 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438779 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4d6b706-1a35-49b4-ae0d-a297613a7546-tmp-dir\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.438809 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438800 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e39891c-14f3-4ac9-948d-dcc8ded4165f-cert\") pod \"ingress-canary-9gnph\" (UID: \"7e39891c-14f3-4ac9-948d-dcc8ded4165f\") " pod="openshift-ingress-canary/ingress-canary-9gnph" Jul 16 10:33:00.439230 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438832 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d8sm\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-kube-api-access-7d8sm\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.439230 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438841 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51dbb08d-7943-4048-90ac-422e0d32f177-ca-trust-extracted\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.439230 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.438877 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-registry-certificates\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.439230 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.439027 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d6b706-1a35-49b4-ae0d-a297613a7546-config-volume\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.439230 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.439168 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4d6b706-1a35-49b4-ae0d-a297613a7546-tmp-dir\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.439611 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.439585 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-registry-certificates\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.439681 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.439660 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-trusted-ca\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.443074 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.443046 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-installation-pull-secrets\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.443188 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.443049 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-image-registry-private-configuration\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.443188 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.443058 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e39891c-14f3-4ac9-948d-dcc8ded4165f-cert\") pod \"ingress-canary-9gnph\" (UID: \"7e39891c-14f3-4ac9-948d-dcc8ded4165f\") " pod="openshift-ingress-canary/ingress-canary-9gnph" Jul 16 10:33:00.448665 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.448575 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-bound-sa-token\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.448838 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.448816 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffj6p\" (UniqueName: \"kubernetes.io/projected/a4d6b706-1a35-49b4-ae0d-a297613a7546-kube-api-access-ffj6p\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.448926 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.448912 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d8sm\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-kube-api-access-7d8sm\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.449132 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.449115 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4n6p\" (UniqueName: \"kubernetes.io/projected/7e39891c-14f3-4ac9-948d-dcc8ded4165f-kube-api-access-t4n6p\") pod \"ingress-canary-9gnph\" (UID: \"7e39891c-14f3-4ac9-948d-dcc8ded4165f\") " pod="openshift-ingress-canary/ingress-canary-9gnph" Jul 16 10:33:00.576205 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.576167 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9gnph" Jul 16 10:33:00.800459 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.800383 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:33:00.803235 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.803212 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xxmn5\"" Jul 16 10:33:00.803360 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.803218 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 10:33:00.944116 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.943934 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:00.944226 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.944195 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:00.944291 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:00.944082 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:33:00.944396 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:00.944334 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls podName:a4d6b706-1a35-49b4-ae0d-a297613a7546 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:01.944316875 +0000 UTC m=+36.623581837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls") pod "dns-default-n8m4p" (UID: "a4d6b706-1a35-49b4-ae0d-a297613a7546") : secret "dns-default-metrics-tls" not found Jul 16 10:33:00.944396 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:00.944278 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:33:00.944396 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:00.944356 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788b9dd6d-fnvz6: secret "image-registry-tls" not found Jul 16 10:33:00.944396 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:00.944377 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls podName:51dbb08d-7943-4048-90ac-422e0d32f177 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:01.944371303 +0000 UTC m=+36.623636260 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls") pod "image-registry-788b9dd6d-fnvz6" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177") : secret "image-registry-tls" not found Jul 16 10:33:00.977309 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:00.977278 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9gnph"] Jul 16 10:33:00.980966 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:33:00.980939 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e39891c_14f3_4ac9_948d_dcc8ded4165f.slice/crio-729f5c9e5257dbcaf32f11bb09dbbd34ebaaa38ee0420b4321e592ef243754bb WatchSource:0}: Error finding container 729f5c9e5257dbcaf32f11bb09dbbd34ebaaa38ee0420b4321e592ef243754bb: Status 404 returned error can't find the container with id 729f5c9e5257dbcaf32f11bb09dbbd34ebaaa38ee0420b4321e592ef243754bb Jul 16 10:33:01.800547 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:01.800464 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:33:01.801021 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:01.800707 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:33:01.803355 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:01.803332 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 10:33:01.803489 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:01.803410 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 10:33:01.803545 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:01.803492 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42vxd\"" Jul 16 10:33:01.804425 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:01.804408 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 10:33:01.951986 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:01.951952 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:01.952165 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:01.952007 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:01.952165 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:01.952103 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:33:01.952165 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:01.952118 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788b9dd6d-fnvz6: secret "image-registry-tls" not found Jul 16 10:33:01.952295 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:01.952167 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:33:01.952295 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:01.952175 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls podName:51dbb08d-7943-4048-90ac-422e0d32f177 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:03.95215644 +0000 UTC m=+38.631421420 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls") pod "image-registry-788b9dd6d-fnvz6" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177") : secret "image-registry-tls" not found Jul 16 10:33:01.952295 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:01.952241 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls podName:a4d6b706-1a35-49b4-ae0d-a297613a7546 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:03.952220878 +0000 UTC m=+38.631485840 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls") pod "dns-default-n8m4p" (UID: "a4d6b706-1a35-49b4-ae0d-a297613a7546") : secret "dns-default-metrics-tls" not found Jul 16 10:33:01.962221 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:01.962188 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9gnph" event={"ID":"7e39891c-14f3-4ac9-948d-dcc8ded4165f","Type":"ContainerStarted","Data":"729f5c9e5257dbcaf32f11bb09dbbd34ebaaa38ee0420b4321e592ef243754bb"} Jul 16 10:33:01.965328 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:01.965303 2581 generic.go:356] "Generic (PLEG): container finished" podID="0fda8e68-706b-4c7d-998d-ba89897e6457" containerID="3d4c9e4bee74a67a0b530693450d661581eaebd19830fc7b70eaf3d658648743" exitCode=0 Jul 16 10:33:01.965464 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:01.965341 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b4t8" event={"ID":"0fda8e68-706b-4c7d-998d-ba89897e6457","Type":"ContainerDied","Data":"3d4c9e4bee74a67a0b530693450d661581eaebd19830fc7b70eaf3d658648743"} Jul 16 10:33:02.970780 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:02.970746 2581 generic.go:356] "Generic (PLEG): container finished" podID="0fda8e68-706b-4c7d-998d-ba89897e6457" containerID="c3e392dbde4dc178c1e2cefe8f7eee34434708e64c415d592569f73e0421bce0" exitCode=0 Jul 16 10:33:02.971145 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:02.970793 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b4t8" event={"ID":"0fda8e68-706b-4c7d-998d-ba89897e6457","Type":"ContainerDied","Data":"c3e392dbde4dc178c1e2cefe8f7eee34434708e64c415d592569f73e0421bce0"} Jul 16 10:33:03.968372 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:03.968211 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:03.968486 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:03.968411 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:03.968486 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:03.968359 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:33:03.968486 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:03.968443 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788b9dd6d-fnvz6: secret "image-registry-tls" not found Jul 16 10:33:03.968612 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:03.968493 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls podName:51dbb08d-7943-4048-90ac-422e0d32f177 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:07.968478299 +0000 UTC m=+42.647743256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls") pod "image-registry-788b9dd6d-fnvz6" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177") : secret "image-registry-tls" not found Jul 16 10:33:03.968612 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:03.968542 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:33:03.968612 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:03.968589 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls podName:a4d6b706-1a35-49b4-ae0d-a297613a7546 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:07.968578055 +0000 UTC m=+42.647843014 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls") pod "dns-default-n8m4p" (UID: "a4d6b706-1a35-49b4-ae0d-a297613a7546") : secret "dns-default-metrics-tls" not found Jul 16 10:33:03.974399 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:03.974371 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9gnph" event={"ID":"7e39891c-14f3-4ac9-948d-dcc8ded4165f","Type":"ContainerStarted","Data":"adc3a42a9fd75ff58fb7507ca6ea2bfc663a61a077587731521b010977bc016b"} Jul 16 10:33:03.977610 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:03.977589 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b4t8" event={"ID":"0fda8e68-706b-4c7d-998d-ba89897e6457","Type":"ContainerStarted","Data":"14ed589164d00b44c93fd3b472829283ad31250b0b6d0b9ec3822c74fb3bbc54"} Jul 16 10:33:04.000810 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:04.000744 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9gnph" podStartSLOduration=1.9010993489999999 podStartE2EDuration="4.000724606s" totalImagesPullingTime="2.099625257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:03.999774496 +0000 UTC m=+38.679039475" watchObservedRunningTime="2026-07-16 10:33:04.000724606 +0000 UTC m=+38.679989590" Jul 16 10:33:06.744169 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.744110 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7b4t8" podStartSLOduration=11.376874947 podStartE2EDuration="40.744091299s" totalImagesPullingTime="29.367216352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:04.043864894 +0000 UTC m=+38.723129873" watchObservedRunningTime="2026-07-16 10:33:06.744091299 +0000 UTC m=+41.423356343" Jul 16 10:33:06.744685 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.744669 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8ts8v"] Jul 16 10:33:06.790271 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.790232 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8ts8v"] Jul 16 10:33:06.790397 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.790355 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.793292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.793266 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 10:33:06.793292 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.793281 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 10:33:06.793490 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.793382 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8nb74\"" Jul 16 10:33:06.794463 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.794445 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 10:33:06.794551 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.794481 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 10:33:06.894186 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.894155 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d675d0fa-bd74-4250-b4a8-989405a36e94-data-volume\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.894342 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.894201 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d675d0fa-bd74-4250-b4a8-989405a36e94-crio-socket\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.894342 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.894223 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.894342 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.894247 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chx6x\" (UniqueName: \"kubernetes.io/projected/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-api-access-chx6x\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.894342 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.894327 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.995550 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.995475 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.995550 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.995517 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chx6x\" (UniqueName: \"kubernetes.io/projected/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-api-access-chx6x\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.995550 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.995539 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.995801 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:06.995604 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 10:33:06.995801 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:06.995708 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm podName:d675d0fa-bd74-4250-b4a8-989405a36e94 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:07.495667338 +0000 UTC m=+42.174932296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8ts8v" (UID: "d675d0fa-bd74-4250-b4a8-989405a36e94") : configmap "kube-rbac-proxy" not found Jul 16 10:33:06.995801 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.995611 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d675d0fa-bd74-4250-b4a8-989405a36e94-data-volume\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.995801 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:06.995725 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 10:33:06.995801 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.995777 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d675d0fa-bd74-4250-b4a8-989405a36e94-crio-socket\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.995967 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:06.995784 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls podName:d675d0fa-bd74-4250-b4a8-989405a36e94 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:07.495768202 +0000 UTC m=+42.175033173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8ts8v" (UID: "d675d0fa-bd74-4250-b4a8-989405a36e94") : secret "insights-runtime-extractor-tls" not found Jul 16 10:33:06.996258 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.996232 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d675d0fa-bd74-4250-b4a8-989405a36e94-crio-socket\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:06.996387 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:06.996141 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d675d0fa-bd74-4250-b4a8-989405a36e94-data-volume\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:07.008427 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:07.008402 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chx6x\" (UniqueName: \"kubernetes.io/projected/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-api-access-chx6x\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:07.500216 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:07.500184 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:07.500401 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:07.500242 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:07.500401 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:07.500318 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 10:33:07.500401 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:07.500390 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm podName:d675d0fa-bd74-4250-b4a8-989405a36e94 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:08.500373233 +0000 UTC m=+43.179638192 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8ts8v" (UID: "d675d0fa-bd74-4250-b4a8-989405a36e94") : configmap "kube-rbac-proxy" not found Jul 16 10:33:07.500524 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:07.500419 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 10:33:07.500524 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:07.500485 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls podName:d675d0fa-bd74-4250-b4a8-989405a36e94 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:08.500472157 +0000 UTC m=+43.179737115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8ts8v" (UID: "d675d0fa-bd74-4250-b4a8-989405a36e94") : secret "insights-runtime-extractor-tls" not found Jul 16 10:33:08.003879 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:08.003843 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:08.004251 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:08.003906 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:08.004251 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:08.003991 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:33:08.004251 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:08.004007 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788b9dd6d-fnvz6: secret "image-registry-tls" not found Jul 16 10:33:08.004251 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:08.004045 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:33:08.004251 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:08.004059 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls podName:51dbb08d-7943-4048-90ac-422e0d32f177 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:16.004044913 +0000 UTC m=+50.683309871 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls") pod "image-registry-788b9dd6d-fnvz6" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177") : secret "image-registry-tls" not found Jul 16 10:33:08.004251 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:08.004089 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls podName:a4d6b706-1a35-49b4-ae0d-a297613a7546 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:16.004076687 +0000 UTC m=+50.683341646 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls") pod "dns-default-n8m4p" (UID: "a4d6b706-1a35-49b4-ae0d-a297613a7546") : secret "dns-default-metrics-tls" not found Jul 16 10:33:08.507509 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:08.507474 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 10:33:08.507654 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:08.507548 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm podName:d675d0fa-bd74-4250-b4a8-989405a36e94 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:10.507534362 +0000 UTC m=+45.186799321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8ts8v" (UID: "d675d0fa-bd74-4250-b4a8-989405a36e94") : configmap "kube-rbac-proxy" not found Jul 16 10:33:08.507654 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:08.507479 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:08.507654 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:08.507610 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:08.507808 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:08.507735 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 10:33:08.507808 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:08.507779 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls podName:d675d0fa-bd74-4250-b4a8-989405a36e94 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:10.507766012 +0000 UTC m=+45.187030970 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8ts8v" (UID: "d675d0fa-bd74-4250-b4a8-989405a36e94") : secret "insights-runtime-extractor-tls" not found Jul 16 10:33:08.738077 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:08.738049 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mtklw_0cb30e43-e55f-4dec-af3f-39960a5946cb/dns-node-resolver/0.log" Jul 16 10:33:09.700719 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.700670 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-zrwc6"] Jul 16 10:33:09.722516 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.722489 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-zrwc6"] Jul 16 10:33:09.722663 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.722581 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" Jul 16 10:33:09.725592 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.725566 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 16 10:33:09.725871 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.725851 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 16 10:33:09.726836 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.726822 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 16 10:33:09.727065 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.727051 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 16 10:33:09.727141 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.727124 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-h8p9x\"" Jul 16 10:33:09.738526 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.738506 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jkm76_b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880/node-ca/0.log" Jul 16 10:33:09.817864 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.817826 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sds4w\" (UniqueName: \"kubernetes.io/projected/c1015088-d821-47ba-9a55-372d951fad78-kube-api-access-sds4w\") pod \"service-ca-79f68b9f4f-zrwc6\" (UID: \"c1015088-d821-47ba-9a55-372d951fad78\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" Jul 16 10:33:09.818026 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.817894 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c1015088-d821-47ba-9a55-372d951fad78-signing-key\") pod \"service-ca-79f68b9f4f-zrwc6\" (UID: \"c1015088-d821-47ba-9a55-372d951fad78\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" Jul 16 10:33:09.818026 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.817955 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c1015088-d821-47ba-9a55-372d951fad78-signing-cabundle\") pod \"service-ca-79f68b9f4f-zrwc6\" (UID: \"c1015088-d821-47ba-9a55-372d951fad78\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" Jul 16 10:33:09.918530 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.918499 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c1015088-d821-47ba-9a55-372d951fad78-signing-cabundle\") pod \"service-ca-79f68b9f4f-zrwc6\" (UID: \"c1015088-d821-47ba-9a55-372d951fad78\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" Jul 16 10:33:09.918615 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.918579 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sds4w\" (UniqueName: \"kubernetes.io/projected/c1015088-d821-47ba-9a55-372d951fad78-kube-api-access-sds4w\") pod \"service-ca-79f68b9f4f-zrwc6\" (UID: \"c1015088-d821-47ba-9a55-372d951fad78\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" Jul 16 10:33:09.918651 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.918624 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c1015088-d821-47ba-9a55-372d951fad78-signing-key\") pod \"service-ca-79f68b9f4f-zrwc6\" (UID: \"c1015088-d821-47ba-9a55-372d951fad78\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" Jul 16 10:33:09.919152 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.919131 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c1015088-d821-47ba-9a55-372d951fad78-signing-cabundle\") pod \"service-ca-79f68b9f4f-zrwc6\" (UID: \"c1015088-d821-47ba-9a55-372d951fad78\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" Jul 16 10:33:09.920923 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.920899 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c1015088-d821-47ba-9a55-372d951fad78-signing-key\") pod \"service-ca-79f68b9f4f-zrwc6\" (UID: \"c1015088-d821-47ba-9a55-372d951fad78\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" Jul 16 10:33:09.931065 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:09.931038 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sds4w\" (UniqueName: \"kubernetes.io/projected/c1015088-d821-47ba-9a55-372d951fad78-kube-api-access-sds4w\") pod \"service-ca-79f68b9f4f-zrwc6\" (UID: \"c1015088-d821-47ba-9a55-372d951fad78\") " pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" Jul 16 10:33:10.032827 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:10.032746 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" Jul 16 10:33:10.204632 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:10.204602 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-zrwc6"] Jul 16 10:33:10.207501 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:33:10.207472 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1015088_d821_47ba_9a55_372d951fad78.slice/crio-34c2f5c4aac052f231b130fc243d16c9635288d2698b014bbe110670457b7ae2 WatchSource:0}: Error finding container 34c2f5c4aac052f231b130fc243d16c9635288d2698b014bbe110670457b7ae2: Status 404 returned error can't find the container with id 34c2f5c4aac052f231b130fc243d16c9635288d2698b014bbe110670457b7ae2 Jul 16 10:33:10.336374 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:10.336350 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9gnph_7e39891c-14f3-4ac9-948d-dcc8ded4165f/serve-healthcheck-canary/0.log" Jul 16 10:33:10.525279 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:10.525243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:10.525529 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:10.525300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:10.525529 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:10.525377 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 10:33:10.525529 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:10.525440 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm podName:d675d0fa-bd74-4250-b4a8-989405a36e94 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:14.525424996 +0000 UTC m=+49.204689954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8ts8v" (UID: "d675d0fa-bd74-4250-b4a8-989405a36e94") : configmap "kube-rbac-proxy" not found Jul 16 10:33:10.525529 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:10.525461 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 10:33:10.525529 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:10.525513 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls podName:d675d0fa-bd74-4250-b4a8-989405a36e94 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:14.525497993 +0000 UTC m=+49.204762953 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8ts8v" (UID: "d675d0fa-bd74-4250-b4a8-989405a36e94") : secret "insights-runtime-extractor-tls" not found Jul 16 10:33:10.996025 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:10.995989 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" event={"ID":"c1015088-d821-47ba-9a55-372d951fad78","Type":"ContainerStarted","Data":"34c2f5c4aac052f231b130fc243d16c9635288d2698b014bbe110670457b7ae2"} Jul 16 10:33:14.004524 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:14.004488 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" event={"ID":"c1015088-d821-47ba-9a55-372d951fad78","Type":"ContainerStarted","Data":"f449bc035af82726c09d6d1b2d7122467290a4d636e7e4dfc4e691468d695e21"} Jul 16 10:33:14.022069 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:14.022014 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-zrwc6" podStartSLOduration=2.279635669 podStartE2EDuration="5.021997839s" totalImagesPullingTime="2.74236217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:14.021588524 +0000 UTC m=+48.700853500" watchObservedRunningTime="2026-07-16 10:33:14.021997839 +0000 UTC m=+48.701262819" Jul 16 10:33:14.558467 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:14.558419 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:14.558661 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:14.558568 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:14.558661 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:14.558602 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 10:33:14.558785 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:14.558706 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls podName:d675d0fa-bd74-4250-b4a8-989405a36e94 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:22.558672119 +0000 UTC m=+57.237937083 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8ts8v" (UID: "d675d0fa-bd74-4250-b4a8-989405a36e94") : secret "insights-runtime-extractor-tls" not found Jul 16 10:33:14.559167 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:14.559147 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d675d0fa-bd74-4250-b4a8-989405a36e94-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:16.067472 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:16.067437 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:16.067867 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:16.067487 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:16.067867 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:16.067582 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 10:33:16.067867 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:16.067600 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-788b9dd6d-fnvz6: secret "image-registry-tls" not found Jul 16 10:33:16.067867 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:16.067646 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls podName:51dbb08d-7943-4048-90ac-422e0d32f177 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:32.067632358 +0000 UTC m=+66.746897317 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls") pod "image-registry-788b9dd6d-fnvz6" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177") : secret "image-registry-tls" not found Jul 16 10:33:16.067867 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:16.067600 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 10:33:16.067867 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:16.067736 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls podName:a4d6b706-1a35-49b4-ae0d-a297613a7546 nodeName:}" failed. No retries permitted until 2026-07-16 10:33:32.067725291 +0000 UTC m=+66.746990249 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls") pod "dns-default-n8m4p" (UID: "a4d6b706-1a35-49b4-ae0d-a297613a7546") : secret "dns-default-metrics-tls" not found Jul 16 10:33:22.619371 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:22.619336 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:22.621565 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:22.621544 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d675d0fa-bd74-4250-b4a8-989405a36e94-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8ts8v\" (UID: \"d675d0fa-bd74-4250-b4a8-989405a36e94\") " pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:22.699747 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:22.699718 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8ts8v" Jul 16 10:33:22.833731 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:22.833700 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8ts8v"] Jul 16 10:33:22.837621 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:33:22.837594 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd675d0fa_bd74_4250_b4a8_989405a36e94.slice/crio-3020565870b51815fa569492825fd97509bd7bf93011c738f5a969c3d47e39fe WatchSource:0}: Error finding container 3020565870b51815fa569492825fd97509bd7bf93011c738f5a969c3d47e39fe: Status 404 returned error can't find the container with id 3020565870b51815fa569492825fd97509bd7bf93011c738f5a969c3d47e39fe Jul 16 10:33:23.025643 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:23.025554 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8ts8v" event={"ID":"d675d0fa-bd74-4250-b4a8-989405a36e94","Type":"ContainerStarted","Data":"3020565870b51815fa569492825fd97509bd7bf93011c738f5a969c3d47e39fe"} Jul 16 10:33:25.959025 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:25.958988 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n6zq" Jul 16 10:33:28.039822 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:28.039740 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8ts8v" event={"ID":"d675d0fa-bd74-4250-b4a8-989405a36e94","Type":"ContainerStarted","Data":"2dbc2036f8299bfb69781e10ebb50ab13202749b67380e5f5dd809b88151eaf8"} Jul 16 10:33:31.599651 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.599616 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:33:31.600209 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.599772 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:33:31.602593 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.602573 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 10:33:31.603579 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.603560 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 10:33:31.612980 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.612952 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24b769d5-b98a-45d0-97fa-e67a68008449-original-pull-secret\") pod \"global-pull-secret-syncer-82c65\" (UID: \"24b769d5-b98a-45d0-97fa-e67a68008449\") " pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:33:31.613259 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.613239 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6cb6750-2018-4424-b782-201c673ee132-metrics-certs\") pod \"network-metrics-daemon-rdp5f\" (UID: \"a6cb6750-2018-4424-b782-201c673ee132\") " pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:33:31.700404 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.700367 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vwqw\" (UniqueName: \"kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw\") pod \"network-check-target-lnndv\" (UID: \"a79fe0b6-6a92-4396-b856-d50f5c5383a3\") " pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:33:31.703379 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.703354 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 10:33:31.713289 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.713254 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xxmn5\"" Jul 16 10:33:31.713289 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.713255 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 10:33:31.721643 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.721614 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rdp5f" Jul 16 10:33:31.724437 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.724413 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vwqw\" (UniqueName: \"kubernetes.io/projected/a79fe0b6-6a92-4396-b856-d50f5c5383a3-kube-api-access-9vwqw\") pod \"network-check-target-lnndv\" (UID: \"a79fe0b6-6a92-4396-b856-d50f5c5383a3\") " pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:33:31.815633 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.815153 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-42vxd\"" Jul 16 10:33:31.818070 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.817885 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-82c65" Jul 16 10:33:31.822540 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.822517 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:33:31.870068 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.869995 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rdp5f"] Jul 16 10:33:31.876766 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:33:31.876718 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6cb6750_2018_4424_b782_201c673ee132.slice/crio-d35c1729269aea5e1569909a549a63ad8fe04037d0a72f1aa17f036c30df1269 WatchSource:0}: Error finding container d35c1729269aea5e1569909a549a63ad8fe04037d0a72f1aa17f036c30df1269: Status 404 returned error can't find the container with id d35c1729269aea5e1569909a549a63ad8fe04037d0a72f1aa17f036c30df1269 Jul 16 10:33:31.977805 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.977772 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-82c65"] Jul 16 10:33:31.981426 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:33:31.981383 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b769d5_b98a_45d0_97fa_e67a68008449.slice/crio-617b0da7d7e5f6ed702b9ccffaa38ea3bd9659b9bf1bd9fb770090b377d7768e WatchSource:0}: Error finding container 617b0da7d7e5f6ed702b9ccffaa38ea3bd9659b9bf1bd9fb770090b377d7768e: Status 404 returned error can't find the container with id 617b0da7d7e5f6ed702b9ccffaa38ea3bd9659b9bf1bd9fb770090b377d7768e Jul 16 10:33:31.997951 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:31.997923 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lnndv"] Jul 16 10:33:32.000778 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:33:32.000749 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79fe0b6_6a92_4396_b856_d50f5c5383a3.slice/crio-09dd136b1c962fd4816fdce219b71d9dbc6c569148e57b8c25293ee99177a204 WatchSource:0}: Error finding container 09dd136b1c962fd4816fdce219b71d9dbc6c569148e57b8c25293ee99177a204: Status 404 returned error can't find the container with id 09dd136b1c962fd4816fdce219b71d9dbc6c569148e57b8c25293ee99177a204 Jul 16 10:33:32.052282 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.052241 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-82c65" event={"ID":"24b769d5-b98a-45d0-97fa-e67a68008449","Type":"ContainerStarted","Data":"617b0da7d7e5f6ed702b9ccffaa38ea3bd9659b9bf1bd9fb770090b377d7768e"} Jul 16 10:33:32.053462 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.053431 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rdp5f" event={"ID":"a6cb6750-2018-4424-b782-201c673ee132","Type":"ContainerStarted","Data":"d35c1729269aea5e1569909a549a63ad8fe04037d0a72f1aa17f036c30df1269"} Jul 16 10:33:32.055309 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.055281 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8ts8v" event={"ID":"d675d0fa-bd74-4250-b4a8-989405a36e94","Type":"ContainerStarted","Data":"91030fa7d749d6e8dc933d260be6d0fc655cc44bd936794fc14898d7de62266d"} Jul 16 10:33:32.056472 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.056448 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lnndv" event={"ID":"a79fe0b6-6a92-4396-b856-d50f5c5383a3","Type":"ContainerStarted","Data":"09dd136b1c962fd4816fdce219b71d9dbc6c569148e57b8c25293ee99177a204"} Jul 16 10:33:32.103248 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.103158 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:32.103407 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.103251 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:32.105845 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.105815 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d6b706-1a35-49b4-ae0d-a297613a7546-metrics-tls\") pod \"dns-default-n8m4p\" (UID: \"a4d6b706-1a35-49b4-ae0d-a297613a7546\") " pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:32.105991 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.105978 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls\") pod \"image-registry-788b9dd6d-fnvz6\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:32.354648 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.354568 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9j5q6\"" Jul 16 10:33:32.362299 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.362272 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:32.371516 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.371496 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x6pbd\"" Jul 16 10:33:32.379152 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.379132 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:32.409500 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.409469 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-788b9dd6d-fnvz6"] Jul 16 10:33:32.439375 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.439339 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j"] Jul 16 10:33:32.463307 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.461742 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.466710 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.466108 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 16 10:33:32.466710 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.466405 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 16 10:33:32.466710 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.466569 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 16 10:33:32.466710 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.466639 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 16 10:33:32.467424 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.467143 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 16 10:33:32.467424 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.467323 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 16 10:33:32.467424 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.467356 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 16 10:33:32.475793 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.475720 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j"] Jul 16 10:33:32.506817 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.506427 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.506817 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.506476 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqnm6\" (UniqueName: \"kubernetes.io/projected/4043155c-6ac6-4e7a-934d-944b5da4e0fd-kube-api-access-qqnm6\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.506817 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.506519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.506817 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.506551 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-ca\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.506817 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.506582 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4043155c-6ac6-4e7a-934d-944b5da4e0fd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.506817 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.506610 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-hub\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.568940 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.568838 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-788b9dd6d-fnvz6"] Jul 16 10:33:32.577370 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:33:32.577338 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51dbb08d_7943_4048_90ac_422e0d32f177.slice/crio-faa669f44380b2ed65896f093903b6a4a4974edd4ea2ab6406793e2a1188e0fa WatchSource:0}: Error finding container faa669f44380b2ed65896f093903b6a4a4974edd4ea2ab6406793e2a1188e0fa: Status 404 returned error can't find the container with id faa669f44380b2ed65896f093903b6a4a4974edd4ea2ab6406793e2a1188e0fa Jul 16 10:33:32.603802 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.603775 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n8m4p"] Jul 16 10:33:32.607494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.607171 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4043155c-6ac6-4e7a-934d-944b5da4e0fd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.607494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.607217 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-hub\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.607494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.607284 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.607494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.607317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqnm6\" (UniqueName: \"kubernetes.io/projected/4043155c-6ac6-4e7a-934d-944b5da4e0fd-kube-api-access-qqnm6\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.607494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.607361 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.607494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.607390 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-ca\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.608042 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.607996 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4043155c-6ac6-4e7a-934d-944b5da4e0fd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.612533 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.612323 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.612777 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:33:32.612746 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d6b706_1a35_49b4_ae0d_a297613a7546.slice/crio-c4e7339fe2e36e8ef8bf3b003723e513284e64b07f426c71595821e94794502e WatchSource:0}: Error finding container c4e7339fe2e36e8ef8bf3b003723e513284e64b07f426c71595821e94794502e: Status 404 returned error can't find the container with id c4e7339fe2e36e8ef8bf3b003723e513284e64b07f426c71595821e94794502e Jul 16 10:33:32.614558 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.614509 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-ca\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.615285 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.615163 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-hub\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.618231 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.618169 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4043155c-6ac6-4e7a-934d-944b5da4e0fd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.618231 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.618168 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqnm6\" (UniqueName: \"kubernetes.io/projected/4043155c-6ac6-4e7a-934d-944b5da4e0fd-kube-api-access-qqnm6\") pod \"cluster-proxy-proxy-agent-cd8d56cbb-47m9j\" (UID: \"4043155c-6ac6-4e7a-934d-944b5da4e0fd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:32.806587 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:32.806554 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" Jul 16 10:33:33.036731 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:33.036650 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j"] Jul 16 10:33:33.045430 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:33:33.045073 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4043155c_6ac6_4e7a_934d_944b5da4e0fd.slice/crio-e5d9b95227a36138895ebb9521bffce4a27f4e0a2f91a6c07d8e813ac8751639 WatchSource:0}: Error finding container e5d9b95227a36138895ebb9521bffce4a27f4e0a2f91a6c07d8e813ac8751639: Status 404 returned error can't find the container with id e5d9b95227a36138895ebb9521bffce4a27f4e0a2f91a6c07d8e813ac8751639 Jul 16 10:33:33.062390 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:33.062357 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n8m4p" event={"ID":"a4d6b706-1a35-49b4-ae0d-a297613a7546","Type":"ContainerStarted","Data":"c4e7339fe2e36e8ef8bf3b003723e513284e64b07f426c71595821e94794502e"} Jul 16 10:33:33.064755 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:33.064469 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" event={"ID":"51dbb08d-7943-4048-90ac-422e0d32f177","Type":"ContainerStarted","Data":"d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef"} Jul 16 10:33:33.064755 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:33.064523 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" event={"ID":"51dbb08d-7943-4048-90ac-422e0d32f177","Type":"ContainerStarted","Data":"faa669f44380b2ed65896f093903b6a4a4974edd4ea2ab6406793e2a1188e0fa"} Jul 16 10:33:33.064755 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:33.064620 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:33.065833 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:33.065811 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" event={"ID":"4043155c-6ac6-4e7a-934d-944b5da4e0fd","Type":"ContainerStarted","Data":"e5d9b95227a36138895ebb9521bffce4a27f4e0a2f91a6c07d8e813ac8751639"} Jul 16 10:33:33.109645 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:33.109583 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" podStartSLOduration=40.109559086 podStartE2EDuration="40.109559086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:33.108437176 +0000 UTC m=+67.787702168" watchObservedRunningTime="2026-07-16 10:33:33.109559086 +0000 UTC m=+67.788824069" Jul 16 10:33:41.099885 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.099849 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-82c65" event={"ID":"24b769d5-b98a-45d0-97fa-e67a68008449","Type":"ContainerStarted","Data":"9f88a1965a560777d793d49ceb9ab7bb756c90eefdfc7302f1238d895bc20912"} Jul 16 10:33:41.101636 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.101613 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rdp5f" event={"ID":"a6cb6750-2018-4424-b782-201c673ee132","Type":"ContainerStarted","Data":"d22654c22f9bd42ce9f96181e7a8e8930f36e05b3dbfc1f33678b98314ab1793"} Jul 16 10:33:41.101773 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.101657 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rdp5f" event={"ID":"a6cb6750-2018-4424-b782-201c673ee132","Type":"ContainerStarted","Data":"463f1a18aef6198206588afd0aa7ec50cc3a33249cb25ce262289ea1aec9ad65"} Jul 16 10:33:41.103537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.103511 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" event={"ID":"4043155c-6ac6-4e7a-934d-944b5da4e0fd","Type":"ContainerStarted","Data":"e4449235f79a671102fa8ca5c1d6584d89b45b65eff9e8d85a361c25562285da"} Jul 16 10:33:41.103610 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.103538 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" event={"ID":"4043155c-6ac6-4e7a-934d-944b5da4e0fd","Type":"ContainerStarted","Data":"f42bca3bdc4dffea9e38be384339e25769e7048043ec779ad1412a74942e0046"} Jul 16 10:33:41.103610 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.103550 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" event={"ID":"4043155c-6ac6-4e7a-934d-944b5da4e0fd","Type":"ContainerStarted","Data":"5c0a557ffd385a4f14cbcfc441464577d3ea8de93b7eaf5de971a0654e9ba007"} Jul 16 10:33:41.105543 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.105522 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8ts8v" event={"ID":"d675d0fa-bd74-4250-b4a8-989405a36e94","Type":"ContainerStarted","Data":"dd6ba8fa0e891b779fdd8088e377c7616ce18a41d24aacb29d2226b1324e4045"} Jul 16 10:33:41.107221 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.107202 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n8m4p" event={"ID":"a4d6b706-1a35-49b4-ae0d-a297613a7546","Type":"ContainerStarted","Data":"c14d90f506bbf545b8b2595ea5668b709845650f6e9b318bf81f78a547e18818"} Jul 16 10:33:41.107276 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.107239 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n8m4p" event={"ID":"a4d6b706-1a35-49b4-ae0d-a297613a7546","Type":"ContainerStarted","Data":"dda6021785402bfa864321ddf360fc9073980b94e58d268ce0da5000a0728818"} Jul 16 10:33:41.107311 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.107301 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:41.108507 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.108489 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lnndv" event={"ID":"a79fe0b6-6a92-4396-b856-d50f5c5383a3","Type":"ContainerStarted","Data":"fa001dbfc81b623a10e630a5da60d46d141c1a61c59dcef5c4ba8f1fa975220b"} Jul 16 10:33:41.108619 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.108606 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:33:41.119396 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.119359 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-82c65" podStartSLOduration=66.581166352 podStartE2EDuration="1m15.119348461s" totalImagesPullingTime="8.538182109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:41.118610452 +0000 UTC m=+75.797875433" watchObservedRunningTime="2026-07-16 10:33:41.119348461 +0000 UTC m=+75.798613439" Jul 16 10:33:41.140495 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.140458 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rdp5f" podStartSLOduration=66.497949226 podStartE2EDuration="1m15.140447459s" totalImagesPullingTime="8.642498233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:41.140014129 +0000 UTC m=+75.819279107" watchObservedRunningTime="2026-07-16 10:33:41.140447459 +0000 UTC m=+75.819712474" Jul 16 10:33:41.162420 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.162379 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lnndv" podStartSLOduration=66.643535874 podStartE2EDuration="1m15.162365407s" totalImagesPullingTime="8.518829533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:41.160863186 +0000 UTC m=+75.840128164" watchObservedRunningTime="2026-07-16 10:33:41.162365407 +0000 UTC m=+75.841630386" Jul 16 10:33:41.181861 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.181817 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n8m4p" podStartSLOduration=33.274544872 podStartE2EDuration="41.181803165s" totalImagesPullingTime="7.907258293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:41.180674856 +0000 UTC m=+75.859939836" watchObservedRunningTime="2026-07-16 10:33:41.181803165 +0000 UTC m=+75.861068144" Jul 16 10:33:41.206748 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.206663 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8d56cbb-47m9j" podStartSLOduration=1.681874542 podStartE2EDuration="9.206647816s" totalImagesPullingTime="7.524773274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:41.206142472 +0000 UTC m=+75.885407451" watchObservedRunningTime="2026-07-16 10:33:41.206647816 +0000 UTC m=+75.885912795" Jul 16 10:33:41.228979 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:41.228936 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8ts8v" podStartSLOduration=17.655409951 podStartE2EDuration="35.228922888s" totalImagesPullingTime="17.573512937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 10:33:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:41.228809661 +0000 UTC m=+75.908074642" watchObservedRunningTime="2026-07-16 10:33:41.228922888 +0000 UTC m=+75.908187867" Jul 16 10:33:48.524062 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.523916 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2t594"] Jul 16 10:33:48.527037 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.527014 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.535963 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.535942 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8gj4m\"" Jul 16 10:33:48.536337 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.536297 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 10:33:48.536411 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.536345 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 10:33:48.536411 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.536388 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 10:33:48.540428 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.540410 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 10:33:48.541312 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.541297 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 10:33:48.562670 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.562645 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 10:33:48.636389 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.636350 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-textfile\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.636389 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.636388 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-tls\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.636608 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.636452 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-sys\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.636608 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.636486 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.636608 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.636525 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-accelerators-collector-config\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.636608 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.636541 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-root\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.636810 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.636631 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-metrics-client-ca\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.636810 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.636678 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jm8m\" (UniqueName: \"kubernetes.io/projected/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-kube-api-access-4jm8m\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.636810 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.636738 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-wtmp\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.737884 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.737845 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-root\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.737884 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.737890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-metrics-client-ca\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.737914 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jm8m\" (UniqueName: \"kubernetes.io/projected/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-kube-api-access-4jm8m\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.737939 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-wtmp\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.737960 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-textfile\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.737974 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-tls\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.737974 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-root\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.737999 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-sys\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.738044 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-sys\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.738042 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.738097 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-wtmp\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738550 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.738101 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-accelerators-collector-config\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738550 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.738337 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-textfile\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738655 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.738549 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-metrics-client-ca\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.738748 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.738678 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-accelerators-collector-config\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.740422 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.740400 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-tls\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.740510 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.740425 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.828857 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.828824 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jm8m\" (UniqueName: \"kubernetes.io/projected/7824e80e-98c8-4eee-b7a5-b7ec0ea48103-kube-api-access-4jm8m\") pod \"node-exporter-2t594\" (UID: \"7824e80e-98c8-4eee-b7a5-b7ec0ea48103\") " pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.836819 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:48.836792 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2t594" Jul 16 10:33:48.847171 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:33:48.847140 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7824e80e_98c8_4eee_b7a5_b7ec0ea48103.slice/crio-5530f6da199a362d4e8e4e578a94466bbf8fcea672223b417882722470307390 WatchSource:0}: Error finding container 5530f6da199a362d4e8e4e578a94466bbf8fcea672223b417882722470307390: Status 404 returned error can't find the container with id 5530f6da199a362d4e8e4e578a94466bbf8fcea672223b417882722470307390 Jul 16 10:33:49.136933 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.136844 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2t594" event={"ID":"7824e80e-98c8-4eee-b7a5-b7ec0ea48103","Type":"ContainerStarted","Data":"5530f6da199a362d4e8e4e578a94466bbf8fcea672223b417882722470307390"} Jul 16 10:33:49.292207 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.292169 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 10:33:49.295128 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.295102 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.299409 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.299368 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 10:33:49.299729 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.299709 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 10:33:49.299815 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.299776 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 10:33:49.301066 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.301043 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vqhtt\"" Jul 16 10:33:49.301635 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.301618 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 10:33:49.302164 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.302145 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 10:33:49.302491 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.302474 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 10:33:49.302624 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.302609 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 10:33:49.303537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.303514 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 10:33:49.303664 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.303521 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 10:33:49.335712 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.315628 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 10:33:49.443402 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443323 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443402 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443362 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443402 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443387 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443623 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443447 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-web-config\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443623 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443489 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443623 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443534 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443623 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443585 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443623 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443608 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443816 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443726 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443816 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443775 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443816 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443799 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4fdc\" (UniqueName: \"kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-kube-api-access-l4fdc\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443914 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443824 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.443914 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.443844 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-out\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.545219 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545174 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4fdc\" (UniqueName: \"kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-kube-api-access-l4fdc\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.545219 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545225 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.545856 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-out\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.545856 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545274 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.545856 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545292 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.545856 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:49.545668 2581 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 16 10:33:49.545856 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545745 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.545856 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:49.545756 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-main-tls podName:3e6f2f1e-2f78-4583-9617-1b37ccf4085e nodeName:}" failed. No retries permitted until 2026-07-16 10:33:50.045734769 +0000 UTC m=+84.724999751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e") : secret "alertmanager-main-tls" not found Jul 16 10:33:49.545856 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545806 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-web-config\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.545856 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545838 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.546245 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.546245 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545925 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.546245 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545948 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.546245 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.545981 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.546245 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.546026 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.546494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.546442 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.547145 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.547115 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.547410 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.547360 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.548449 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.548411 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.548982 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.548945 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.548982 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.548971 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-out\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.549423 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.549386 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.549423 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.549397 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.549880 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.549855 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.550670 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.550642 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-web-config\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.550925 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.550908 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:49.557193 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:49.557175 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4fdc\" (UniqueName: \"kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-kube-api-access-l4fdc\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:50.050900 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:50.050809 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:50.053331 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:50.053302 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:50.141469 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:50.141427 2581 generic.go:356] "Generic (PLEG): container finished" podID="7824e80e-98c8-4eee-b7a5-b7ec0ea48103" containerID="5312310bfd7965866a9a119a51fae089bad79ee18290163ea0dba22540c11a91" exitCode=0 Jul 16 10:33:50.141668 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:50.141537 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2t594" event={"ID":"7824e80e-98c8-4eee-b7a5-b7ec0ea48103","Type":"ContainerDied","Data":"5312310bfd7965866a9a119a51fae089bad79ee18290163ea0dba22540c11a91"} Jul 16 10:33:50.211085 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:50.211061 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:33:50.356807 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:50.356786 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 10:33:50.358265 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:33:50.358230 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e6f2f1e_2f78_4583_9617_1b37ccf4085e.slice/crio-e590401909f201bdfb7884e04f09a117ac263e04d03df0dba8e405918844f6fd WatchSource:0}: Error finding container e590401909f201bdfb7884e04f09a117ac263e04d03df0dba8e405918844f6fd: Status 404 returned error can't find the container with id e590401909f201bdfb7884e04f09a117ac263e04d03df0dba8e405918844f6fd Jul 16 10:33:51.115237 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:51.115209 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n8m4p" Jul 16 10:33:51.147359 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:51.147328 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2t594" event={"ID":"7824e80e-98c8-4eee-b7a5-b7ec0ea48103","Type":"ContainerStarted","Data":"954b218b18418032fb8645b97b9db32d2ac5dd401322743105c59afbf34660d7"} Jul 16 10:33:51.147534 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:51.147388 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2t594" event={"ID":"7824e80e-98c8-4eee-b7a5-b7ec0ea48103","Type":"ContainerStarted","Data":"3489c5297ef35396c076bdd12f074c650310601cde6956704490b1fb15b0bee7"} Jul 16 10:33:51.148454 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:51.148426 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerStarted","Data":"e590401909f201bdfb7884e04f09a117ac263e04d03df0dba8e405918844f6fd"} Jul 16 10:33:52.153364 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:52.153327 2581 generic.go:356] "Generic (PLEG): container finished" podID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerID="d0ffe35893ea8715367dfaa7194c47a85e7e6fda86e5453a044760797e0b2b5d" exitCode=0 Jul 16 10:33:52.153874 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:52.153404 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerDied","Data":"d0ffe35893ea8715367dfaa7194c47a85e7e6fda86e5453a044760797e0b2b5d"} Jul 16 10:33:52.186049 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:52.185999 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2t594" podStartSLOduration=3.302336314 podStartE2EDuration="4.185984009s" totalImagesPullingTime="819.071001ms" totalInitContainerRuntime="64.576694ms" isStatefulPod=true podCreationTimestamp="2026-07-16 10:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:51.183233573 +0000 UTC m=+85.862498553" watchObservedRunningTime="2026-07-16 10:33:52.185984009 +0000 UTC m=+86.865248988" Jul 16 10:33:53.071701 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:53.071660 2581 patch_prober.go:28] interesting pod/image-registry-788b9dd6d-fnvz6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 10:33:53.071941 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:53.071793 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" podUID="51dbb08d-7943-4048-90ac-422e0d32f177" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 10:33:54.164005 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:54.163972 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerStarted","Data":"e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749"} Jul 16 10:33:54.164382 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:54.164039 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerStarted","Data":"5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16"} Jul 16 10:33:54.164382 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:54.164065 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerStarted","Data":"2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc"} Jul 16 10:33:54.164382 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:54.164088 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerStarted","Data":"7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f"} Jul 16 10:33:54.164382 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:54.164120 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerStarted","Data":"5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd"} Jul 16 10:33:55.173964 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:55.173928 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerStarted","Data":"f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837"} Jul 16 10:33:55.209108 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:55.208921 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.969504751 podStartE2EDuration="6.208903082s" totalImagesPullingTime="3.222849913s" totalInitContainerRuntime="16.548418ms" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:33:55.208202034 +0000 UTC m=+89.887467010" watchObservedRunningTime="2026-07-16 10:33:55.208903082 +0000 UTC m=+89.888168063" Jul 16 10:33:58.097373 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.097323 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" podUID="51dbb08d-7943-4048-90ac-422e0d32f177" containerName="registry" containerID="cri-o://d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef" gracePeriod=30 Jul 16 10:33:58.825329 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.825306 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:58.923898 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.923865 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-registry-certificates\") pod \"51dbb08d-7943-4048-90ac-422e0d32f177\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " Jul 16 10:33:58.924123 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.923908 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-trusted-ca\") pod \"51dbb08d-7943-4048-90ac-422e0d32f177\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " Jul 16 10:33:58.924123 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.923933 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-bound-sa-token\") pod \"51dbb08d-7943-4048-90ac-422e0d32f177\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " Jul 16 10:33:58.924123 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.923970 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-installation-pull-secrets\") pod \"51dbb08d-7943-4048-90ac-422e0d32f177\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " Jul 16 10:33:58.924123 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.924004 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-image-registry-private-configuration\") pod \"51dbb08d-7943-4048-90ac-422e0d32f177\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " Jul 16 10:33:58.924123 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.924032 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7d8sm\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-kube-api-access-7d8sm\") pod \"51dbb08d-7943-4048-90ac-422e0d32f177\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " Jul 16 10:33:58.924123 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.924054 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls\") pod \"51dbb08d-7943-4048-90ac-422e0d32f177\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " Jul 16 10:33:58.924123 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.924095 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51dbb08d-7943-4048-90ac-422e0d32f177-ca-trust-extracted\") pod \"51dbb08d-7943-4048-90ac-422e0d32f177\" (UID: \"51dbb08d-7943-4048-90ac-422e0d32f177\") " Jul 16 10:33:58.924427 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.924382 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "51dbb08d-7943-4048-90ac-422e0d32f177" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:33:58.924532 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.924503 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "51dbb08d-7943-4048-90ac-422e0d32f177" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:33:58.926468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.926438 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "51dbb08d-7943-4048-90ac-422e0d32f177" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:33:58.926602 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.926463 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "51dbb08d-7943-4048-90ac-422e0d32f177" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:33:58.926602 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.926560 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "51dbb08d-7943-4048-90ac-422e0d32f177" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:33:58.926602 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.926574 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-kube-api-access-7d8sm" (OuterVolumeSpecName: "kube-api-access-7d8sm") pod "51dbb08d-7943-4048-90ac-422e0d32f177" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177"). InnerVolumeSpecName "kube-api-access-7d8sm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:33:58.926884 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.926868 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "51dbb08d-7943-4048-90ac-422e0d32f177" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:33:58.932663 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:58.932639 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51dbb08d-7943-4048-90ac-422e0d32f177-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "51dbb08d-7943-4048-90ac-422e0d32f177" (UID: "51dbb08d-7943-4048-90ac-422e0d32f177"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:33:59.025493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.025415 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-registry-certificates\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:33:59.025493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.025443 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51dbb08d-7943-4048-90ac-422e0d32f177-trusted-ca\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:33:59.025493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.025452 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-bound-sa-token\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:33:59.025493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.025460 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-installation-pull-secrets\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:33:59.025493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.025470 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51dbb08d-7943-4048-90ac-422e0d32f177-image-registry-private-configuration\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:33:59.025493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.025486 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7d8sm\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-kube-api-access-7d8sm\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:33:59.025493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.025494 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51dbb08d-7943-4048-90ac-422e0d32f177-registry-tls\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:33:59.025493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.025502 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51dbb08d-7943-4048-90ac-422e0d32f177-ca-trust-extracted\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:33:59.188983 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.188950 2581 generic.go:356] "Generic (PLEG): container finished" podID="51dbb08d-7943-4048-90ac-422e0d32f177" containerID="d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef" exitCode=0 Jul 16 10:33:59.189368 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.189001 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" Jul 16 10:33:59.189368 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.189003 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" event={"ID":"51dbb08d-7943-4048-90ac-422e0d32f177","Type":"ContainerDied","Data":"d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef"} Jul 16 10:33:59.189368 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.189081 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-788b9dd6d-fnvz6" event={"ID":"51dbb08d-7943-4048-90ac-422e0d32f177","Type":"ContainerDied","Data":"faa669f44380b2ed65896f093903b6a4a4974edd4ea2ab6406793e2a1188e0fa"} Jul 16 10:33:59.189368 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.189108 2581 scope.go:117] "RemoveContainer" containerID="d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef" Jul 16 10:33:59.197267 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.197247 2581 scope.go:117] "RemoveContainer" containerID="d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef" Jul 16 10:33:59.197504 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:33:59.197482 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef\": container with ID starting with d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef not found: ID does not exist" containerID="d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef" Jul 16 10:33:59.197581 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.197528 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef"} err="failed to get container status \"d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef\": rpc error: code = NotFound desc = could not find container \"d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef\": container with ID starting with d60ff1f41f353da2a537760c06e8930674048681b220e4609373f7d8b4ec65ef not found: ID does not exist" Jul 16 10:33:59.213819 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.213796 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-788b9dd6d-fnvz6"] Jul 16 10:33:59.228720 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.228703 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-788b9dd6d-fnvz6"] Jul 16 10:33:59.687911 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.687878 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-748585757b-n44km"] Jul 16 10:33:59.688230 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.688202 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51dbb08d-7943-4048-90ac-422e0d32f177" containerName="registry" Jul 16 10:33:59.688230 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.688222 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="51dbb08d-7943-4048-90ac-422e0d32f177" containerName="registry" Jul 16 10:33:59.688363 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.688288 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="51dbb08d-7943-4048-90ac-422e0d32f177" containerName="registry" Jul 16 10:33:59.692705 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.692666 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.699928 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.699913 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 10:33:59.700930 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.700907 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 10:33:59.701032 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.700907 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 10:33:59.701158 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.701143 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 10:33:59.701248 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.701232 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 10:33:59.701298 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.701252 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 10:33:59.701298 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.701293 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 10:33:59.701411 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.701397 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qp7fd\"" Jul 16 10:33:59.706634 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.706619 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 10:33:59.708418 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.708400 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-748585757b-n44km"] Jul 16 10:33:59.804373 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.804332 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51dbb08d-7943-4048-90ac-422e0d32f177" path="/var/lib/kubelet/pods/51dbb08d-7943-4048-90ac-422e0d32f177/volumes" Jul 16 10:33:59.830813 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.830784 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-oauth-config\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.830961 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.830817 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-oauth-serving-cert\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.830961 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.830846 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpddr\" (UniqueName: \"kubernetes.io/projected/fea67f05-3506-4760-a20b-704321d6fe5f-kube-api-access-tpddr\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.830961 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.830924 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-console-config\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.831074 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.830979 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-trusted-ca-bundle\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.831074 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.831031 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-serving-cert\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.831074 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.831046 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-service-ca\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.932212 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.932175 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-oauth-config\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.932212 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.932214 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-oauth-serving-cert\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.932379 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.932231 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpddr\" (UniqueName: \"kubernetes.io/projected/fea67f05-3506-4760-a20b-704321d6fe5f-kube-api-access-tpddr\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.932379 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.932275 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-console-config\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.932379 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.932302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-trusted-ca-bundle\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.932379 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.932343 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-serving-cert\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.932541 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.932397 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-service-ca\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.933161 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.933135 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-console-config\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.933161 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.933156 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-service-ca\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.933413 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.933389 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-oauth-serving-cert\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.933753 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.933732 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-trusted-ca-bundle\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.934655 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.934634 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-serving-cert\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.934777 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.934761 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-oauth-config\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:33:59.941595 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:33:59.941545 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpddr\" (UniqueName: \"kubernetes.io/projected/fea67f05-3506-4760-a20b-704321d6fe5f-kube-api-access-tpddr\") pod \"console-748585757b-n44km\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " pod="openshift-console/console-748585757b-n44km" Jul 16 10:34:00.001395 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:00.001362 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748585757b-n44km" Jul 16 10:34:00.129820 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:00.129792 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-748585757b-n44km"] Jul 16 10:34:00.133046 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:34:00.133019 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea67f05_3506_4760_a20b_704321d6fe5f.slice/crio-9868eb579f74fe08d60c58282a11ae9a1793190f3db300cebcb837fabf0e0e41 WatchSource:0}: Error finding container 9868eb579f74fe08d60c58282a11ae9a1793190f3db300cebcb837fabf0e0e41: Status 404 returned error can't find the container with id 9868eb579f74fe08d60c58282a11ae9a1793190f3db300cebcb837fabf0e0e41 Jul 16 10:34:00.193794 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:00.193730 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748585757b-n44km" event={"ID":"fea67f05-3506-4760-a20b-704321d6fe5f","Type":"ContainerStarted","Data":"9868eb579f74fe08d60c58282a11ae9a1793190f3db300cebcb837fabf0e0e41"} Jul 16 10:34:03.207975 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:03.207900 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748585757b-n44km" event={"ID":"fea67f05-3506-4760-a20b-704321d6fe5f","Type":"ContainerStarted","Data":"045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654"} Jul 16 10:34:03.243317 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:03.243270 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-748585757b-n44km" podStartSLOduration=1.482363779 podStartE2EDuration="4.243250955s" totalImagesPullingTime="2.760887176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:34:03.242590162 +0000 UTC m=+97.921855137" watchObservedRunningTime="2026-07-16 10:34:03.243250955 +0000 UTC m=+97.922515933" Jul 16 10:34:09.596057 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:09.596026 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57fd58c5f8-58d67"] Jul 16 10:34:09.877732 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:09.877633 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57fd58c5f8-58d67"] Jul 16 10:34:09.877876 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:09.877789 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:09.907562 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:09.907531 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-oauth-config\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:09.907562 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:09.907561 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-oauth-serving-cert\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:09.907765 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:09.907582 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-config\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:09.907765 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:09.907666 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-service-ca\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:09.907765 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:09.907726 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-trusted-ca-bundle\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:09.907765 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:09.907753 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkr2g\" (UniqueName: \"kubernetes.io/projected/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-kube-api-access-rkr2g\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:09.907888 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:09.907777 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-serving-cert\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.002309 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.002272 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-748585757b-n44km" Jul 16 10:34:10.002429 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.002347 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-748585757b-n44km" Jul 16 10:34:10.007124 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.007102 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-748585757b-n44km" Jul 16 10:34:10.008360 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.008344 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-oauth-config\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.008460 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.008368 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-oauth-serving-cert\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.008460 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.008391 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-config\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.008553 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.008514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-service-ca\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.008602 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.008548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-trusted-ca-bundle\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.008658 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.008614 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkr2g\" (UniqueName: \"kubernetes.io/projected/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-kube-api-access-rkr2g\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.008733 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.008656 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-serving-cert\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.009372 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.009348 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-config\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.009468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.009366 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-service-ca\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.009572 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.009532 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-trusted-ca-bundle\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.009751 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.009737 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-oauth-serving-cert\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.011044 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.011025 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-oauth-config\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.011102 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.011073 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-serving-cert\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.021129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.021106 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkr2g\" (UniqueName: \"kubernetes.io/projected/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-kube-api-access-rkr2g\") pod \"console-57fd58c5f8-58d67\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.187118 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.187037 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:10.236955 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.236928 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-748585757b-n44km" Jul 16 10:34:10.358560 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:10.358530 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57fd58c5f8-58d67"] Jul 16 10:34:10.361606 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:34:10.361578 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5831bf_aab1_44c9_9bea_8ee790f0cf60.slice/crio-2edf798bc871d2c041eeac6995cea6e5eb0edb3f8d4d2451f6526a675d2a7ffd WatchSource:0}: Error finding container 2edf798bc871d2c041eeac6995cea6e5eb0edb3f8d4d2451f6526a675d2a7ffd: Status 404 returned error can't find the container with id 2edf798bc871d2c041eeac6995cea6e5eb0edb3f8d4d2451f6526a675d2a7ffd Jul 16 10:34:11.237625 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:11.237592 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fd58c5f8-58d67" event={"ID":"8c5831bf-aab1-44c9-9bea-8ee790f0cf60","Type":"ContainerStarted","Data":"01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c"} Jul 16 10:34:11.238035 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:11.237645 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fd58c5f8-58d67" event={"ID":"8c5831bf-aab1-44c9-9bea-8ee790f0cf60","Type":"ContainerStarted","Data":"2edf798bc871d2c041eeac6995cea6e5eb0edb3f8d4d2451f6526a675d2a7ffd"} Jul 16 10:34:11.269813 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:11.269758 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57fd58c5f8-58d67" podStartSLOduration=2.269740215 podStartE2EDuration="2.269740215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:34:11.269389148 +0000 UTC m=+105.948654120" watchObservedRunningTime="2026-07-16 10:34:11.269740215 +0000 UTC m=+105.949005195" Jul 16 10:34:12.114854 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:12.114823 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lnndv" Jul 16 10:34:20.187314 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:20.187275 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:20.187792 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:20.187333 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:20.191922 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:20.191902 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:20.270151 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:20.270126 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:34:20.323366 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:20.323336 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-748585757b-n44km"] Jul 16 10:34:45.347815 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.347751 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-748585757b-n44km" podUID="fea67f05-3506-4760-a20b-704321d6fe5f" containerName="console" containerID="cri-o://045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654" gracePeriod=15 Jul 16 10:34:45.578103 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.578080 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748585757b-n44km" Jul 16 10:34:45.672830 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.672733 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-trusted-ca-bundle\") pod \"fea67f05-3506-4760-a20b-704321d6fe5f\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " Jul 16 10:34:45.672830 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.672782 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-serving-cert\") pod \"fea67f05-3506-4760-a20b-704321d6fe5f\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " Jul 16 10:34:45.672830 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.672809 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-service-ca\") pod \"fea67f05-3506-4760-a20b-704321d6fe5f\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " Jul 16 10:34:45.673098 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.672987 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-oauth-config\") pod \"fea67f05-3506-4760-a20b-704321d6fe5f\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " Jul 16 10:34:45.673098 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.673048 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-console-config\") pod \"fea67f05-3506-4760-a20b-704321d6fe5f\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " Jul 16 10:34:45.673098 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.673091 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-oauth-serving-cert\") pod \"fea67f05-3506-4760-a20b-704321d6fe5f\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " Jul 16 10:34:45.673263 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.673136 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpddr\" (UniqueName: \"kubernetes.io/projected/fea67f05-3506-4760-a20b-704321d6fe5f-kube-api-access-tpddr\") pod \"fea67f05-3506-4760-a20b-704321d6fe5f\" (UID: \"fea67f05-3506-4760-a20b-704321d6fe5f\") " Jul 16 10:34:45.673317 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.673132 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fea67f05-3506-4760-a20b-704321d6fe5f" (UID: "fea67f05-3506-4760-a20b-704321d6fe5f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:34:45.673317 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.673169 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-service-ca" (OuterVolumeSpecName: "service-ca") pod "fea67f05-3506-4760-a20b-704321d6fe5f" (UID: "fea67f05-3506-4760-a20b-704321d6fe5f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:34:45.673433 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.673412 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-trusted-ca-bundle\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:34:45.673491 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.673439 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-service-ca\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:34:45.673491 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.673449 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-console-config" (OuterVolumeSpecName: "console-config") pod "fea67f05-3506-4760-a20b-704321d6fe5f" (UID: "fea67f05-3506-4760-a20b-704321d6fe5f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:34:45.673589 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.673513 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fea67f05-3506-4760-a20b-704321d6fe5f" (UID: "fea67f05-3506-4760-a20b-704321d6fe5f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:34:45.675131 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.675106 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fea67f05-3506-4760-a20b-704321d6fe5f" (UID: "fea67f05-3506-4760-a20b-704321d6fe5f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:34:45.675232 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.675149 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fea67f05-3506-4760-a20b-704321d6fe5f" (UID: "fea67f05-3506-4760-a20b-704321d6fe5f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:34:45.675232 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.675199 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea67f05-3506-4760-a20b-704321d6fe5f-kube-api-access-tpddr" (OuterVolumeSpecName: "kube-api-access-tpddr") pod "fea67f05-3506-4760-a20b-704321d6fe5f" (UID: "fea67f05-3506-4760-a20b-704321d6fe5f"). InnerVolumeSpecName "kube-api-access-tpddr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:34:45.774585 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.774550 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-console-config\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:34:45.774585 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.774576 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fea67f05-3506-4760-a20b-704321d6fe5f-oauth-serving-cert\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:34:45.774585 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.774585 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpddr\" (UniqueName: \"kubernetes.io/projected/fea67f05-3506-4760-a20b-704321d6fe5f-kube-api-access-tpddr\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:34:45.774585 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.774594 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-serving-cert\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:34:45.774878 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:45.774602 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fea67f05-3506-4760-a20b-704321d6fe5f-console-oauth-config\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:34:46.359971 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:46.359938 2581 generic.go:356] "Generic (PLEG): container finished" podID="fea67f05-3506-4760-a20b-704321d6fe5f" containerID="045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654" exitCode=0 Jul 16 10:34:46.359971 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:46.359974 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748585757b-n44km" event={"ID":"fea67f05-3506-4760-a20b-704321d6fe5f","Type":"ContainerDied","Data":"045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654"} Jul 16 10:34:46.360409 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:46.359982 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748585757b-n44km" Jul 16 10:34:46.360409 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:46.360010 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748585757b-n44km" event={"ID":"fea67f05-3506-4760-a20b-704321d6fe5f","Type":"ContainerDied","Data":"9868eb579f74fe08d60c58282a11ae9a1793190f3db300cebcb837fabf0e0e41"} Jul 16 10:34:46.360409 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:46.360029 2581 scope.go:117] "RemoveContainer" containerID="045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654" Jul 16 10:34:46.367813 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:46.367782 2581 scope.go:117] "RemoveContainer" containerID="045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654" Jul 16 10:34:46.368046 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:34:46.368026 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654\": container with ID starting with 045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654 not found: ID does not exist" containerID="045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654" Jul 16 10:34:46.368086 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:46.368060 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654"} err="failed to get container status \"045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654\": rpc error: code = NotFound desc = could not find container \"045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654\": container with ID starting with 045f7be36b0f7700765f6b7889f426c7c249484ed0afeef0f078e1d8aa6e7654 not found: ID does not exist" Jul 16 10:34:46.379818 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:46.379795 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-748585757b-n44km"] Jul 16 10:34:46.383258 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:46.383235 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-748585757b-n44km"] Jul 16 10:34:47.804440 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:34:47.804397 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fea67f05-3506-4760-a20b-704321d6fe5f" path="/var/lib/kubelet/pods/fea67f05-3506-4760-a20b-704321d6fe5f/volumes" Jul 16 10:35:08.729565 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:08.729528 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 10:35:08.730041 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:08.729852 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="alertmanager" containerID="cri-o://5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd" gracePeriod=120 Jul 16 10:35:08.730041 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:08.729907 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy-metric" containerID="cri-o://e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749" gracePeriod=120 Jul 16 10:35:08.730041 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:08.729931 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="prom-label-proxy" containerID="cri-o://f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837" gracePeriod=120 Jul 16 10:35:08.730041 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:08.729973 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy-web" containerID="cri-o://2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc" gracePeriod=120 Jul 16 10:35:08.730041 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:08.730027 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy" containerID="cri-o://5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16" gracePeriod=120 Jul 16 10:35:08.730269 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:08.729979 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="config-reloader" containerID="cri-o://7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f" gracePeriod=120 Jul 16 10:35:09.441005 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:09.440972 2581 generic.go:356] "Generic (PLEG): container finished" podID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerID="f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837" exitCode=0 Jul 16 10:35:09.441005 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:09.440995 2581 generic.go:356] "Generic (PLEG): container finished" podID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerID="e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749" exitCode=0 Jul 16 10:35:09.441005 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:09.441001 2581 generic.go:356] "Generic (PLEG): container finished" podID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerID="5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16" exitCode=0 Jul 16 10:35:09.441005 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:09.441007 2581 generic.go:356] "Generic (PLEG): container finished" podID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerID="7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f" exitCode=0 Jul 16 10:35:09.441005 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:09.441012 2581 generic.go:356] "Generic (PLEG): container finished" podID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerID="5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd" exitCode=0 Jul 16 10:35:09.441286 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:09.441039 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerDied","Data":"f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837"} Jul 16 10:35:09.441286 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:09.441096 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerDied","Data":"e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749"} Jul 16 10:35:09.441286 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:09.441121 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerDied","Data":"5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16"} Jul 16 10:35:09.441286 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:09.441144 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerDied","Data":"7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f"} Jul 16 10:35:09.441286 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:09.441167 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerDied","Data":"5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd"} Jul 16 10:35:09.971816 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:09.971796 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.047558 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047485 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-volume\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.047558 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047534 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-web\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.047785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047571 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-metrics-client-ca\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.047785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047607 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-main-db\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.047785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047633 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4fdc\" (UniqueName: \"kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-kube-api-access-l4fdc\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.047785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047670 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.047785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047712 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-trusted-ca-bundle\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.047785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047741 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-out\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.047785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047779 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-main-tls\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.048113 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047814 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-web-config\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.048113 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047850 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-cluster-tls-config\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.048113 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047875 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.048113 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.047896 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-tls-assets\") pod \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\" (UID: \"3e6f2f1e-2f78-4583-9617-1b37ccf4085e\") " Jul 16 10:35:10.048113 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.048018 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:35:10.048424 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.048136 2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-metrics-client-ca\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.048498 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.048472 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:35:10.048679 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.048646 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:35:10.051785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.051742 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:35:10.051904 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.051786 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:35:10.051904 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.051817 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-volume" (OuterVolumeSpecName: "config-volume") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:35:10.051991 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.051900 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:35:10.052173 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.052123 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-kube-api-access-l4fdc" (OuterVolumeSpecName: "kube-api-access-l4fdc") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "kube-api-access-l4fdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:35:10.052539 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.052511 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:35:10.052647 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.052545 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-out" (OuterVolumeSpecName: "config-out") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:35:10.052846 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.052815 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:35:10.055901 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.055878 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:35:10.061622 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.061589 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-web-config" (OuterVolumeSpecName: "web-config") pod "3e6f2f1e-2f78-4583-9617-1b37ccf4085e" (UID: "3e6f2f1e-2f78-4583-9617-1b37ccf4085e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:35:10.148961 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.148921 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-main-db\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.148961 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.148952 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4fdc\" (UniqueName: \"kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-kube-api-access-l4fdc\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.148961 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.148962 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.149201 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.148973 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.149201 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.148984 2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-out\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.149201 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.148991 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-main-tls\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.149201 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.149020 2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-web-config\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.149201 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.149028 2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-cluster-tls-config\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.149201 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.149036 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.149201 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.149047 2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-tls-assets\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.149201 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.149055 2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-config-volume\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.149201 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.149062 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e6f2f1e-2f78-4583-9617-1b37ccf4085e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:35:10.447849 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.447806 2581 generic.go:356] "Generic (PLEG): container finished" podID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerID="2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc" exitCode=0 Jul 16 10:35:10.448035 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.447864 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerDied","Data":"2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc"} Jul 16 10:35:10.448035 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.447887 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.448035 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.447935 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e6f2f1e-2f78-4583-9617-1b37ccf4085e","Type":"ContainerDied","Data":"e590401909f201bdfb7884e04f09a117ac263e04d03df0dba8e405918844f6fd"} Jul 16 10:35:10.448035 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.447977 2581 scope.go:117] "RemoveContainer" containerID="f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837" Jul 16 10:35:10.456184 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.456163 2581 scope.go:117] "RemoveContainer" containerID="e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749" Jul 16 10:35:10.462816 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.462799 2581 scope.go:117] "RemoveContainer" containerID="5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16" Jul 16 10:35:10.468945 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.468928 2581 scope.go:117] "RemoveContainer" containerID="2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc" Jul 16 10:35:10.473911 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.473890 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 10:35:10.475877 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.475861 2581 scope.go:117] "RemoveContainer" containerID="7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f" Jul 16 10:35:10.480702 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.480559 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 10:35:10.484870 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.484850 2581 scope.go:117] "RemoveContainer" containerID="5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd" Jul 16 10:35:10.507005 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.506982 2581 scope.go:117] "RemoveContainer" containerID="d0ffe35893ea8715367dfaa7194c47a85e7e6fda86e5453a044760797e0b2b5d" Jul 16 10:35:10.514010 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.513989 2581 scope.go:117] "RemoveContainer" containerID="f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837" Jul 16 10:35:10.514279 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:35:10.514258 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837\": container with ID starting with f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837 not found: ID does not exist" containerID="f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837" Jul 16 10:35:10.514345 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.514286 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837"} err="failed to get container status \"f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837\": rpc error: code = NotFound desc = could not find container \"f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837\": container with ID starting with f87d776b85076dd03d40da28ef6cc03074f160cf1dc6f4ab2ad306aebe0de837 not found: ID does not exist" Jul 16 10:35:10.514345 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.514305 2581 scope.go:117] "RemoveContainer" containerID="e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749" Jul 16 10:35:10.514573 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:35:10.514555 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749\": container with ID starting with e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749 not found: ID does not exist" containerID="e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749" Jul 16 10:35:10.514636 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.514583 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749"} err="failed to get container status \"e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749\": rpc error: code = NotFound desc = could not find container \"e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749\": container with ID starting with e5d83334e0d2d4259d37bfa58c684ae5731cf097f93979ee056692e2a7373749 not found: ID does not exist" Jul 16 10:35:10.514636 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.514606 2581 scope.go:117] "RemoveContainer" containerID="5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16" Jul 16 10:35:10.514862 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:35:10.514839 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16\": container with ID starting with 5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16 not found: ID does not exist" containerID="5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16" Jul 16 10:35:10.514934 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.514859 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16"} err="failed to get container status \"5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16\": rpc error: code = NotFound desc = could not find container \"5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16\": container with ID starting with 5dabaebec1c12237c22399aa06bec03b4482282ed4028982bcabc1e276422c16 not found: ID does not exist" Jul 16 10:35:10.514934 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.514878 2581 scope.go:117] "RemoveContainer" containerID="2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc" Jul 16 10:35:10.515153 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:35:10.515136 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc\": container with ID starting with 2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc not found: ID does not exist" containerID="2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc" Jul 16 10:35:10.515219 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.515167 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc"} err="failed to get container status \"2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc\": rpc error: code = NotFound desc = could not find container \"2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc\": container with ID starting with 2f521455442f2bef5375b3fd4a62e7c510be29c33be8a63762d8799a65e2f6dc not found: ID does not exist" Jul 16 10:35:10.515219 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.515188 2581 scope.go:117] "RemoveContainer" containerID="7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f" Jul 16 10:35:10.515434 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:35:10.515414 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f\": container with ID starting with 7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f not found: ID does not exist" containerID="7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f" Jul 16 10:35:10.515487 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.515438 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f"} err="failed to get container status \"7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f\": rpc error: code = NotFound desc = could not find container \"7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f\": container with ID starting with 7c77ba5fdd47b097403ba170e3a1af1049a889bd9e8703aa0d7d4dea5ee27e4f not found: ID does not exist" Jul 16 10:35:10.515487 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.515453 2581 scope.go:117] "RemoveContainer" containerID="5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd" Jul 16 10:35:10.515815 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:35:10.515793 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd\": container with ID starting with 5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd not found: ID does not exist" containerID="5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd" Jul 16 10:35:10.515902 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.515819 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd"} err="failed to get container status \"5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd\": rpc error: code = NotFound desc = could not find container \"5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd\": container with ID starting with 5abfdf5899470b2babfd2a21b024618ee010d1fe3e607bf26c4783e484a750dd not found: ID does not exist" Jul 16 10:35:10.515902 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.515833 2581 scope.go:117] "RemoveContainer" containerID="d0ffe35893ea8715367dfaa7194c47a85e7e6fda86e5453a044760797e0b2b5d" Jul 16 10:35:10.516095 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:35:10.516070 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0ffe35893ea8715367dfaa7194c47a85e7e6fda86e5453a044760797e0b2b5d\": container with ID starting with d0ffe35893ea8715367dfaa7194c47a85e7e6fda86e5453a044760797e0b2b5d not found: ID does not exist" containerID="d0ffe35893ea8715367dfaa7194c47a85e7e6fda86e5453a044760797e0b2b5d" Jul 16 10:35:10.516192 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.516099 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0ffe35893ea8715367dfaa7194c47a85e7e6fda86e5453a044760797e0b2b5d"} err="failed to get container status \"d0ffe35893ea8715367dfaa7194c47a85e7e6fda86e5453a044760797e0b2b5d\": rpc error: code = NotFound desc = could not find container \"d0ffe35893ea8715367dfaa7194c47a85e7e6fda86e5453a044760797e0b2b5d\": container with ID starting with d0ffe35893ea8715367dfaa7194c47a85e7e6fda86e5453a044760797e0b2b5d not found: ID does not exist" Jul 16 10:35:10.519151 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519130 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 10:35:10.519363 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519352 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy-metric" Jul 16 10:35:10.519401 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519365 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy-metric" Jul 16 10:35:10.519401 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519373 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy-web" Jul 16 10:35:10.519401 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519378 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy-web" Jul 16 10:35:10.519401 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519387 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fea67f05-3506-4760-a20b-704321d6fe5f" containerName="console" Jul 16 10:35:10.519401 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519396 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fea67f05-3506-4760-a20b-704321d6fe5f" containerName="console" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519408 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="init-config-reloader" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519413 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="init-config-reloader" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519419 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="alertmanager" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519425 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="alertmanager" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519431 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="config-reloader" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519436 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="config-reloader" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519441 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="prom-label-proxy" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519446 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="prom-label-proxy" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519458 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519464 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519501 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="config-reloader" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519508 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="alertmanager" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519513 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fea67f05-3506-4760-a20b-704321d6fe5f" containerName="console" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519521 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519527 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy-metric" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519532 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="kube-rbac-proxy-web" Jul 16 10:35:10.519537 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.519538 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" containerName="prom-label-proxy" Jul 16 10:35:10.523988 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.523975 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.529946 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.529771 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 10:35:10.529946 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.529774 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 10:35:10.530364 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.530347 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 10:35:10.531136 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.530868 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 10:35:10.531136 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.530884 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 10:35:10.531136 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.530901 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 10:35:10.531136 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.530913 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 10:35:10.531136 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.530877 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 10:35:10.531136 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.531146 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vqhtt\"" Jul 16 10:35:10.537678 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.537661 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 10:35:10.550919 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.550892 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 10:35:10.653785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.653745 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.653785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.653790 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb84ae97-4f67-41f0-920f-183be30c6d39-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.653997 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.653816 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.653997 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.653924 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb84ae97-4f67-41f0-920f-183be30c6d39-config-out\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.653997 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.653955 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb84ae97-4f67-41f0-920f-183be30c6d39-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.653997 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.653977 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f29ss\" (UniqueName: \"kubernetes.io/projected/fb84ae97-4f67-41f0-920f-183be30c6d39-kube-api-access-f29ss\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.654121 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.654030 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.654121 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.654048 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-web-config\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.654121 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.654092 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb84ae97-4f67-41f0-920f-183be30c6d39-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.654212 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.654126 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.654212 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.654158 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.654212 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.654177 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb84ae97-4f67-41f0-920f-183be30c6d39-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.654212 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.654206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.754775 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.754660 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb84ae97-4f67-41f0-920f-183be30c6d39-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.754775 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.754758 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.755004 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.754812 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.755004 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.754837 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb84ae97-4f67-41f0-920f-183be30c6d39-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.755004 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.754861 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.755004 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.754892 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.755004 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.754918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb84ae97-4f67-41f0-920f-183be30c6d39-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.755004 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.754942 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.755004 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.754987 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb84ae97-4f67-41f0-920f-183be30c6d39-config-out\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.755310 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.755011 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb84ae97-4f67-41f0-920f-183be30c6d39-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.755310 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.755038 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f29ss\" (UniqueName: \"kubernetes.io/projected/fb84ae97-4f67-41f0-920f-183be30c6d39-kube-api-access-f29ss\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.755310 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.755087 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.755310 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.755111 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-web-config\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.756499 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.755603 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb84ae97-4f67-41f0-920f-183be30c6d39-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.756499 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.756215 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb84ae97-4f67-41f0-920f-183be30c6d39-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.756499 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.756422 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb84ae97-4f67-41f0-920f-183be30c6d39-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.757967 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.757850 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-web-config\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.757967 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.757904 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb84ae97-4f67-41f0-920f-183be30c6d39-config-out\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.757967 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.757918 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.758114 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.757983 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.758114 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.758021 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.758114 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.758092 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb84ae97-4f67-41f0-920f-183be30c6d39-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.758809 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.758786 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.758880 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.758868 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.759636 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.759620 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb84ae97-4f67-41f0-920f-183be30c6d39-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.767330 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.767311 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f29ss\" (UniqueName: \"kubernetes.io/projected/fb84ae97-4f67-41f0-920f-183be30c6d39-kube-api-access-f29ss\") pod \"alertmanager-main-0\" (UID: \"fb84ae97-4f67-41f0-920f-183be30c6d39\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.832635 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.832599 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 10:35:10.990939 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:10.990910 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 10:35:10.994314 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:35:10.994285 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb84ae97_4f67_41f0_920f_183be30c6d39.slice/crio-fab1c8366223b0ab32952cf46f2dc6c3fe7179fac3f7bbd100efbbc3c11dc8f9 WatchSource:0}: Error finding container fab1c8366223b0ab32952cf46f2dc6c3fe7179fac3f7bbd100efbbc3c11dc8f9: Status 404 returned error can't find the container with id fab1c8366223b0ab32952cf46f2dc6c3fe7179fac3f7bbd100efbbc3c11dc8f9 Jul 16 10:35:11.453345 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:11.453292 2581 generic.go:356] "Generic (PLEG): container finished" podID="fb84ae97-4f67-41f0-920f-183be30c6d39" containerID="1fc88c15b560481b28abbc12434614a6b2a0e84d96703afc2d8d43e3f25a088c" exitCode=0 Jul 16 10:35:11.453485 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:11.453379 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb84ae97-4f67-41f0-920f-183be30c6d39","Type":"ContainerDied","Data":"1fc88c15b560481b28abbc12434614a6b2a0e84d96703afc2d8d43e3f25a088c"} Jul 16 10:35:11.453485 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:11.453431 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb84ae97-4f67-41f0-920f-183be30c6d39","Type":"ContainerStarted","Data":"fab1c8366223b0ab32952cf46f2dc6c3fe7179fac3f7bbd100efbbc3c11dc8f9"} Jul 16 10:35:11.804883 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:11.804852 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e6f2f1e-2f78-4583-9617-1b37ccf4085e" path="/var/lib/kubelet/pods/3e6f2f1e-2f78-4583-9617-1b37ccf4085e/volumes" Jul 16 10:35:12.460231 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:12.460195 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb84ae97-4f67-41f0-920f-183be30c6d39","Type":"ContainerStarted","Data":"f3eb4b782a65e201e636c89c7fcd627955dfe874fc21ec079ccf8481b49a41e7"} Jul 16 10:35:12.460593 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:12.460275 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb84ae97-4f67-41f0-920f-183be30c6d39","Type":"ContainerStarted","Data":"35389759fccd46ebc070fa9c5c66b6cc0f70fc998375cec9c24be8c092bc5d2d"} Jul 16 10:35:12.460593 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:12.460317 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb84ae97-4f67-41f0-920f-183be30c6d39","Type":"ContainerStarted","Data":"09071f46c8281bdfc54b3a57c559ea51769f90073cb5b4c7fc79ed3952305ec3"} Jul 16 10:35:12.460593 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:12.460353 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb84ae97-4f67-41f0-920f-183be30c6d39","Type":"ContainerStarted","Data":"29b01f86446d120ad0b3f93a5d4bc1bb708de4ebffb2c3adee09607b36a8b476"} Jul 16 10:35:12.460593 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:12.460388 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb84ae97-4f67-41f0-920f-183be30c6d39","Type":"ContainerStarted","Data":"371810af50e4304883dbbc8a829ead3a3cc5b8f352be9acf295ecd8c4f8ffdbb"} Jul 16 10:35:12.460593 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:12.460421 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb84ae97-4f67-41f0-920f-183be30c6d39","Type":"ContainerStarted","Data":"5f989c3f8c79bc636cb489b897a2d5b4e4851f15ad76268a58749d76646fd962"} Jul 16 10:35:12.501084 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:12.501036 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.485438674 podStartE2EDuration="2.501019385s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.580711ms" isStatefulPod=false podCreationTimestamp="2026-07-16 10:35:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:35:12.500209941 +0000 UTC m=+167.179474919" watchObservedRunningTime="2026-07-16 10:35:12.501019385 +0000 UTC m=+167.180284364" Jul 16 10:35:53.657788 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:35:53.657754 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57fd58c5f8-58d67"] Jul 16 10:36:18.677042 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:18.676979 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57fd58c5f8-58d67" podUID="8c5831bf-aab1-44c9-9bea-8ee790f0cf60" containerName="console" containerID="cri-o://01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c" gracePeriod=15 Jul 16 10:36:18.915933 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:18.915911 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:36:19.041071 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.040959 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-config\") pod \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " Jul 16 10:36:19.041071 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.041027 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-oauth-config\") pod \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " Jul 16 10:36:19.041071 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.041052 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkr2g\" (UniqueName: \"kubernetes.io/projected/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-kube-api-access-rkr2g\") pod \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " Jul 16 10:36:19.041359 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.041166 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-trusted-ca-bundle\") pod \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " Jul 16 10:36:19.041359 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.041206 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-serving-cert\") pod \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " Jul 16 10:36:19.041359 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.041251 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-service-ca\") pod \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " Jul 16 10:36:19.041359 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.041286 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-oauth-serving-cert\") pod \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\" (UID: \"8c5831bf-aab1-44c9-9bea-8ee790f0cf60\") " Jul 16 10:36:19.041555 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.041476 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-config" (OuterVolumeSpecName: "console-config") pod "8c5831bf-aab1-44c9-9bea-8ee790f0cf60" (UID: "8c5831bf-aab1-44c9-9bea-8ee790f0cf60"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:36:19.041616 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.041592 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-service-ca" (OuterVolumeSpecName: "service-ca") pod "8c5831bf-aab1-44c9-9bea-8ee790f0cf60" (UID: "8c5831bf-aab1-44c9-9bea-8ee790f0cf60"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:36:19.041680 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.041580 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8c5831bf-aab1-44c9-9bea-8ee790f0cf60" (UID: "8c5831bf-aab1-44c9-9bea-8ee790f0cf60"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:36:19.041815 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.041754 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8c5831bf-aab1-44c9-9bea-8ee790f0cf60" (UID: "8c5831bf-aab1-44c9-9bea-8ee790f0cf60"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:36:19.043170 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.043145 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8c5831bf-aab1-44c9-9bea-8ee790f0cf60" (UID: "8c5831bf-aab1-44c9-9bea-8ee790f0cf60"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:36:19.043270 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.043191 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8c5831bf-aab1-44c9-9bea-8ee790f0cf60" (UID: "8c5831bf-aab1-44c9-9bea-8ee790f0cf60"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:36:19.043270 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.043214 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-kube-api-access-rkr2g" (OuterVolumeSpecName: "kube-api-access-rkr2g") pod "8c5831bf-aab1-44c9-9bea-8ee790f0cf60" (UID: "8c5831bf-aab1-44c9-9bea-8ee790f0cf60"). InnerVolumeSpecName "kube-api-access-rkr2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:36:19.142391 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.142348 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-oauth-serving-cert\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:19.142391 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.142385 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-config\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:19.142391 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.142398 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-oauth-config\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:19.142589 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.142410 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkr2g\" (UniqueName: \"kubernetes.io/projected/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-kube-api-access-rkr2g\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:19.142589 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.142422 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-trusted-ca-bundle\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:19.142589 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.142432 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-console-serving-cert\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:19.142589 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.142443 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c5831bf-aab1-44c9-9bea-8ee790f0cf60-service-ca\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:19.691978 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.691942 2581 generic.go:356] "Generic (PLEG): container finished" podID="8c5831bf-aab1-44c9-9bea-8ee790f0cf60" containerID="01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c" exitCode=0 Jul 16 10:36:19.692377 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.691984 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57fd58c5f8-58d67" Jul 16 10:36:19.692377 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.692022 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fd58c5f8-58d67" event={"ID":"8c5831bf-aab1-44c9-9bea-8ee790f0cf60","Type":"ContainerDied","Data":"01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c"} Jul 16 10:36:19.692377 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.692066 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57fd58c5f8-58d67" event={"ID":"8c5831bf-aab1-44c9-9bea-8ee790f0cf60","Type":"ContainerDied","Data":"2edf798bc871d2c041eeac6995cea6e5eb0edb3f8d4d2451f6526a675d2a7ffd"} Jul 16 10:36:19.692377 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.692084 2581 scope.go:117] "RemoveContainer" containerID="01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c" Jul 16 10:36:19.700718 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.700681 2581 scope.go:117] "RemoveContainer" containerID="01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c" Jul 16 10:36:19.700947 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:36:19.700932 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c\": container with ID starting with 01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c not found: ID does not exist" containerID="01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c" Jul 16 10:36:19.700995 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.700954 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c"} err="failed to get container status \"01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c\": rpc error: code = NotFound desc = could not find container \"01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c\": container with ID starting with 01e683838022e54ee638946add387a898fa3c153f0647cad51e18ac11677145c not found: ID does not exist" Jul 16 10:36:19.713492 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.713471 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57fd58c5f8-58d67"] Jul 16 10:36:19.722161 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.722141 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57fd58c5f8-58d67"] Jul 16 10:36:19.804457 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:19.804431 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c5831bf-aab1-44c9-9bea-8ee790f0cf60" path="/var/lib/kubelet/pods/8c5831bf-aab1-44c9-9bea-8ee790f0cf60/volumes" Jul 16 10:36:20.370742 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.370711 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2"] Jul 16 10:36:20.370955 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.370944 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c5831bf-aab1-44c9-9bea-8ee790f0cf60" containerName="console" Jul 16 10:36:20.370995 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.370957 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c5831bf-aab1-44c9-9bea-8ee790f0cf60" containerName="console" Jul 16 10:36:20.371031 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.371001 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c5831bf-aab1-44c9-9bea-8ee790f0cf60" containerName="console" Jul 16 10:36:20.374996 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.374978 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:20.378040 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.378022 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 10:36:20.378175 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.378158 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9xvqh\"" Jul 16 10:36:20.379284 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.379269 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 10:36:20.384840 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.384716 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2"] Jul 16 10:36:20.451500 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.451472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:20.451626 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.451515 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvblr\" (UniqueName: \"kubernetes.io/projected/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-kube-api-access-jvblr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:20.451626 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.451550 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:20.552802 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.552774 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:20.552915 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.552822 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvblr\" (UniqueName: \"kubernetes.io/projected/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-kube-api-access-jvblr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:20.552915 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.552875 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:20.553287 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.553266 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:20.553323 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.553277 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:20.562991 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.562963 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvblr\" (UniqueName: \"kubernetes.io/projected/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-kube-api-access-jvblr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:20.683885 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.683802 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:20.806190 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:20.806163 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2"] Jul 16 10:36:20.809546 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:36:20.809515 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f68b0d_c528_4834_9d9e_4d4ceb3bf42a.slice/crio-d5b793f20466f39498bd98745e3fd102cbb3a24635a50277ce8cade1b6589e0c WatchSource:0}: Error finding container d5b793f20466f39498bd98745e3fd102cbb3a24635a50277ce8cade1b6589e0c: Status 404 returned error can't find the container with id d5b793f20466f39498bd98745e3fd102cbb3a24635a50277ce8cade1b6589e0c Jul 16 10:36:21.702608 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:21.702570 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" event={"ID":"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a","Type":"ContainerStarted","Data":"d5b793f20466f39498bd98745e3fd102cbb3a24635a50277ce8cade1b6589e0c"} Jul 16 10:36:27.724810 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:27.724773 2581 generic.go:356] "Generic (PLEG): container finished" podID="13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" containerID="55aab76fdadd1285c8750b351ee375181be6effc1cf8c5390f280949a2a79a77" exitCode=0 Jul 16 10:36:27.725268 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:27.724855 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" event={"ID":"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a","Type":"ContainerDied","Data":"55aab76fdadd1285c8750b351ee375181be6effc1cf8c5390f280949a2a79a77"} Jul 16 10:36:29.733786 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:29.733756 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" event={"ID":"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a","Type":"ContainerStarted","Data":"7824bf7ab759fb87276566a424f273e9c92d9981a8aae51bba493ec9faf766c4"} Jul 16 10:36:30.739351 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:30.739321 2581 generic.go:356] "Generic (PLEG): container finished" podID="13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" containerID="7824bf7ab759fb87276566a424f273e9c92d9981a8aae51bba493ec9faf766c4" exitCode=0 Jul 16 10:36:30.739753 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:30.739366 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" event={"ID":"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a","Type":"ContainerDied","Data":"7824bf7ab759fb87276566a424f273e9c92d9981a8aae51bba493ec9faf766c4"} Jul 16 10:36:37.768230 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:37.768143 2581 generic.go:356] "Generic (PLEG): container finished" podID="13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" containerID="3b208d2ea7a761147a5bd472b1b487072b46d0f1743db26b240165dd72a22382" exitCode=0 Jul 16 10:36:37.768230 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:37.768204 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" event={"ID":"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a","Type":"ContainerDied","Data":"3b208d2ea7a761147a5bd472b1b487072b46d0f1743db26b240165dd72a22382"} Jul 16 10:36:38.894370 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:38.894348 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:38.999705 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:38.999659 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-bundle\") pod \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " Jul 16 10:36:38.999868 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:38.999782 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvblr\" (UniqueName: \"kubernetes.io/projected/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-kube-api-access-jvblr\") pod \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " Jul 16 10:36:38.999868 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:38.999811 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-util\") pod \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\" (UID: \"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a\") " Jul 16 10:36:39.000357 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:39.000336 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-bundle" (OuterVolumeSpecName: "bundle") pod "13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" (UID: "13f68b0d-c528-4834-9d9e-4d4ceb3bf42a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:36:39.001940 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:39.001909 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-kube-api-access-jvblr" (OuterVolumeSpecName: "kube-api-access-jvblr") pod "13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" (UID: "13f68b0d-c528-4834-9d9e-4d4ceb3bf42a"). InnerVolumeSpecName "kube-api-access-jvblr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:36:39.005521 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:39.005497 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-util" (OuterVolumeSpecName: "util") pod "13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" (UID: "13f68b0d-c528-4834-9d9e-4d4ceb3bf42a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:36:39.100806 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:39.100773 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvblr\" (UniqueName: \"kubernetes.io/projected/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-kube-api-access-jvblr\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:39.100806 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:39.100806 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-util\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:39.100806 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:39.100816 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13f68b0d-c528-4834-9d9e-4d4ceb3bf42a-bundle\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:39.777010 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:39.776973 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" event={"ID":"13f68b0d-c528-4834-9d9e-4d4ceb3bf42a","Type":"ContainerDied","Data":"d5b793f20466f39498bd98745e3fd102cbb3a24635a50277ce8cade1b6589e0c"} Jul 16 10:36:39.777010 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:39.777017 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5b793f20466f39498bd98745e3fd102cbb3a24635a50277ce8cade1b6589e0c" Jul 16 10:36:39.777204 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:39.776989 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sdbg2" Jul 16 10:36:43.373969 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.373940 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8"] Jul 16 10:36:43.374337 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.374159 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" containerName="pull" Jul 16 10:36:43.374337 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.374169 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" containerName="pull" Jul 16 10:36:43.374337 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.374182 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" containerName="util" Jul 16 10:36:43.374337 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.374187 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" containerName="util" Jul 16 10:36:43.374337 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.374195 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" containerName="extract" Jul 16 10:36:43.374337 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.374201 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" containerName="extract" Jul 16 10:36:43.374337 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.374242 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="13f68b0d-c528-4834-9d9e-4d4ceb3bf42a" containerName="extract" Jul 16 10:36:43.377377 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.377360 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8" Jul 16 10:36:43.380245 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.380228 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:36:43.380444 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.380426 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 10:36:43.380532 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.380425 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-k9fdl\"" Jul 16 10:36:43.390551 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.390530 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8"] Jul 16 10:36:43.531347 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.531310 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blcjn\" (UniqueName: \"kubernetes.io/projected/f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca-kube-api-access-blcjn\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnjq8\" (UID: \"f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8" Jul 16 10:36:43.531515 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.531368 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnjq8\" (UID: \"f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8" Jul 16 10:36:43.632605 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.632519 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blcjn\" (UniqueName: \"kubernetes.io/projected/f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca-kube-api-access-blcjn\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnjq8\" (UID: \"f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8" Jul 16 10:36:43.632605 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.632575 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnjq8\" (UID: \"f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8" Jul 16 10:36:43.632924 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.632908 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnjq8\" (UID: \"f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8" Jul 16 10:36:43.642103 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.642082 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blcjn\" (UniqueName: \"kubernetes.io/projected/f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca-kube-api-access-blcjn\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnjq8\" (UID: \"f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8" Jul 16 10:36:43.685988 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.685957 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8" Jul 16 10:36:43.893644 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:43.893624 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8"] Jul 16 10:36:43.895131 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:36:43.895108 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf091bcd0_2cc9_435b_af4a_dd8b9f8e4cca.slice/crio-f7dee37810ea4c8a4b014fa7d9bc52bf47cea04e868730a3f16f94660de038cc WatchSource:0}: Error finding container f7dee37810ea4c8a4b014fa7d9bc52bf47cea04e868730a3f16f94660de038cc: Status 404 returned error can't find the container with id f7dee37810ea4c8a4b014fa7d9bc52bf47cea04e868730a3f16f94660de038cc Jul 16 10:36:44.794349 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:44.794311 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8" event={"ID":"f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca","Type":"ContainerStarted","Data":"f7dee37810ea4c8a4b014fa7d9bc52bf47cea04e868730a3f16f94660de038cc"} Jul 16 10:36:45.798772 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:45.798675 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8" event={"ID":"f091bcd0-2cc9-435b-af4a-dd8b9f8e4cca","Type":"ContainerStarted","Data":"e047bc8e4d01da72db0ffc390cccd25b249c0966caba244feea7e9dec9098f97"} Jul 16 10:36:45.827972 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:45.827922 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnjq8" podStartSLOduration=1.336112182 podStartE2EDuration="2.827907622s" totalImagesPullingTime="1.49179544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:36:45.826394298 +0000 UTC m=+260.505659279" watchObservedRunningTime="2026-07-16 10:36:45.827907622 +0000 UTC m=+260.507172600" Jul 16 10:36:47.269400 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.269321 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn"] Jul 16 10:36:47.271435 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.271420 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:47.280423 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.280403 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 10:36:47.280531 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.280493 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9xvqh\"" Jul 16 10:36:47.289682 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.289668 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 10:36:47.441511 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.441476 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn"] Jul 16 10:36:47.459665 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.459634 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:47.459824 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.459671 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j62mb\" (UniqueName: \"kubernetes.io/projected/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-kube-api-access-j62mb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:47.459824 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.459794 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:47.560549 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.560468 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:47.560549 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.560510 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j62mb\" (UniqueName: \"kubernetes.io/projected/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-kube-api-access-j62mb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:47.560733 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.560548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:47.560868 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.560847 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:47.560935 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.560919 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:47.573634 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.573610 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j62mb\" (UniqueName: \"kubernetes.io/projected/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-kube-api-access-j62mb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:47.580441 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.580415 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:47.712174 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.712149 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn"] Jul 16 10:36:47.714666 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:36:47.714637 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bafa26c_a31c_4a45_aea6_ed4f4a275e85.slice/crio-6e21c9135b514750962449b63ed620e9a85228e87efba04f3dfca5174fd19f00 WatchSource:0}: Error finding container 6e21c9135b514750962449b63ed620e9a85228e87efba04f3dfca5174fd19f00: Status 404 returned error can't find the container with id 6e21c9135b514750962449b63ed620e9a85228e87efba04f3dfca5174fd19f00 Jul 16 10:36:47.806366 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.806323 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" event={"ID":"9bafa26c-a31c-4a45-aea6-ed4f4a275e85","Type":"ContainerStarted","Data":"fc287082cdcc594e232c073860dd6e01688e363a8485222d1cddf5e12c24a718"} Jul 16 10:36:47.806366 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:47.806369 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" event={"ID":"9bafa26c-a31c-4a45-aea6-ed4f4a275e85","Type":"ContainerStarted","Data":"6e21c9135b514750962449b63ed620e9a85228e87efba04f3dfca5174fd19f00"} Jul 16 10:36:48.138596 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.138559 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n6cbk"] Jul 16 10:36:48.140439 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.140418 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" Jul 16 10:36:48.143702 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.143669 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 10:36:48.143795 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.143669 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-n8lpz\"" Jul 16 10:36:48.144953 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.144939 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 10:36:48.157015 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.156994 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n6cbk"] Jul 16 10:36:48.265249 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.265217 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbpdw\" (UniqueName: \"kubernetes.io/projected/13c92e7a-d734-4a64-ace1-fbda1fdd3f1c-kube-api-access-rbpdw\") pod \"cert-manager-webhook-6486ffd997-n6cbk\" (UID: \"13c92e7a-d734-4a64-ace1-fbda1fdd3f1c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" Jul 16 10:36:48.265249 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.265252 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13c92e7a-d734-4a64-ace1-fbda1fdd3f1c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n6cbk\" (UID: \"13c92e7a-d734-4a64-ace1-fbda1fdd3f1c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" Jul 16 10:36:48.366336 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.366297 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbpdw\" (UniqueName: \"kubernetes.io/projected/13c92e7a-d734-4a64-ace1-fbda1fdd3f1c-kube-api-access-rbpdw\") pod \"cert-manager-webhook-6486ffd997-n6cbk\" (UID: \"13c92e7a-d734-4a64-ace1-fbda1fdd3f1c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" Jul 16 10:36:48.366336 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.366340 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13c92e7a-d734-4a64-ace1-fbda1fdd3f1c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n6cbk\" (UID: \"13c92e7a-d734-4a64-ace1-fbda1fdd3f1c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" Jul 16 10:36:48.379111 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.379075 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13c92e7a-d734-4a64-ace1-fbda1fdd3f1c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n6cbk\" (UID: \"13c92e7a-d734-4a64-ace1-fbda1fdd3f1c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" Jul 16 10:36:48.379239 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.379224 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbpdw\" (UniqueName: \"kubernetes.io/projected/13c92e7a-d734-4a64-ace1-fbda1fdd3f1c-kube-api-access-rbpdw\") pod \"cert-manager-webhook-6486ffd997-n6cbk\" (UID: \"13c92e7a-d734-4a64-ace1-fbda1fdd3f1c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" Jul 16 10:36:48.448597 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.448524 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" Jul 16 10:36:48.569682 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.569660 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n6cbk"] Jul 16 10:36:48.571841 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:36:48.571813 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13c92e7a_d734_4a64_ace1_fbda1fdd3f1c.slice/crio-e351e5406545292331202e44ff19a5f22695928c92f96559b26d9dd3bb90c8d2 WatchSource:0}: Error finding container e351e5406545292331202e44ff19a5f22695928c92f96559b26d9dd3bb90c8d2: Status 404 returned error can't find the container with id e351e5406545292331202e44ff19a5f22695928c92f96559b26d9dd3bb90c8d2 Jul 16 10:36:48.813011 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.811848 2581 generic.go:356] "Generic (PLEG): container finished" podID="9bafa26c-a31c-4a45-aea6-ed4f4a275e85" containerID="fc287082cdcc594e232c073860dd6e01688e363a8485222d1cddf5e12c24a718" exitCode=0 Jul 16 10:36:48.813011 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.811963 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" event={"ID":"9bafa26c-a31c-4a45-aea6-ed4f4a275e85","Type":"ContainerDied","Data":"fc287082cdcc594e232c073860dd6e01688e363a8485222d1cddf5e12c24a718"} Jul 16 10:36:48.814524 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:48.814502 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" event={"ID":"13c92e7a-d734-4a64-ace1-fbda1fdd3f1c","Type":"ContainerStarted","Data":"e351e5406545292331202e44ff19a5f22695928c92f96559b26d9dd3bb90c8d2"} Jul 16 10:36:51.829935 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:51.829896 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" event={"ID":"9bafa26c-a31c-4a45-aea6-ed4f4a275e85","Type":"ContainerStarted","Data":"f347933f102bc022514270eae024fb9fd16e161c9131712dba4e77ca5616aeaf"} Jul 16 10:36:51.831416 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:51.831389 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" event={"ID":"13c92e7a-d734-4a64-ace1-fbda1fdd3f1c","Type":"ContainerStarted","Data":"26ba68614816d8f92fc021959f0ff0674bae300da8de889ecd047bc12b839b01"} Jul 16 10:36:51.831562 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:51.831522 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" Jul 16 10:36:51.867611 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:51.867561 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" podStartSLOduration=0.727721191 podStartE2EDuration="3.867546504s" totalImagesPullingTime="3.139825313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:36:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:36:51.866382471 +0000 UTC m=+266.545647462" watchObservedRunningTime="2026-07-16 10:36:51.867546504 +0000 UTC m=+266.546811483" Jul 16 10:36:52.836474 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:52.836441 2581 generic.go:356] "Generic (PLEG): container finished" podID="9bafa26c-a31c-4a45-aea6-ed4f4a275e85" containerID="f347933f102bc022514270eae024fb9fd16e161c9131712dba4e77ca5616aeaf" exitCode=0 Jul 16 10:36:52.836962 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:52.836527 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" event={"ID":"9bafa26c-a31c-4a45-aea6-ed4f4a275e85","Type":"ContainerDied","Data":"f347933f102bc022514270eae024fb9fd16e161c9131712dba4e77ca5616aeaf"} Jul 16 10:36:53.842132 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:53.842098 2581 generic.go:356] "Generic (PLEG): container finished" podID="9bafa26c-a31c-4a45-aea6-ed4f4a275e85" containerID="d376ef92d22f2b94cde98259e9158ac10845d4db805122c4edc37d21ab59063e" exitCode=0 Jul 16 10:36:53.842580 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:53.842180 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" event={"ID":"9bafa26c-a31c-4a45-aea6-ed4f4a275e85","Type":"ContainerDied","Data":"d376ef92d22f2b94cde98259e9158ac10845d4db805122c4edc37d21ab59063e"} Jul 16 10:36:54.966321 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:54.966297 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:55.018555 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.018534 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-bundle\") pod \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " Jul 16 10:36:55.018702 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.018568 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j62mb\" (UniqueName: \"kubernetes.io/projected/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-kube-api-access-j62mb\") pod \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " Jul 16 10:36:55.018702 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.018592 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-util\") pod \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\" (UID: \"9bafa26c-a31c-4a45-aea6-ed4f4a275e85\") " Jul 16 10:36:55.018950 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.018929 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-bundle" (OuterVolumeSpecName: "bundle") pod "9bafa26c-a31c-4a45-aea6-ed4f4a275e85" (UID: "9bafa26c-a31c-4a45-aea6-ed4f4a275e85"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:36:55.020562 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.020528 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-kube-api-access-j62mb" (OuterVolumeSpecName: "kube-api-access-j62mb") pod "9bafa26c-a31c-4a45-aea6-ed4f4a275e85" (UID: "9bafa26c-a31c-4a45-aea6-ed4f4a275e85"). InnerVolumeSpecName "kube-api-access-j62mb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:36:55.023322 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.023300 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-util" (OuterVolumeSpecName: "util") pod "9bafa26c-a31c-4a45-aea6-ed4f4a275e85" (UID: "9bafa26c-a31c-4a45-aea6-ed4f4a275e85"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:36:55.119094 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.119028 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-bundle\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:55.119094 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.119058 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j62mb\" (UniqueName: \"kubernetes.io/projected/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-kube-api-access-j62mb\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:55.119094 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.119068 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bafa26c-a31c-4a45-aea6-ed4f4a275e85-util\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:36:55.851113 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.851081 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" event={"ID":"9bafa26c-a31c-4a45-aea6-ed4f4a275e85","Type":"ContainerDied","Data":"6e21c9135b514750962449b63ed620e9a85228e87efba04f3dfca5174fd19f00"} Jul 16 10:36:55.851265 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.851130 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e21c9135b514750962449b63ed620e9a85228e87efba04f3dfca5174fd19f00" Jul 16 10:36:55.851265 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:55.851087 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frb7jn" Jul 16 10:36:57.838826 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:36:57.838799 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-n6cbk" Jul 16 10:37:00.715989 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.715958 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff"] Jul 16 10:37:00.716340 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.716187 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bafa26c-a31c-4a45-aea6-ed4f4a275e85" containerName="util" Jul 16 10:37:00.716340 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.716196 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bafa26c-a31c-4a45-aea6-ed4f4a275e85" containerName="util" Jul 16 10:37:00.716340 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.716203 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bafa26c-a31c-4a45-aea6-ed4f4a275e85" containerName="pull" Jul 16 10:37:00.716340 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.716208 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bafa26c-a31c-4a45-aea6-ed4f4a275e85" containerName="pull" Jul 16 10:37:00.716340 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.716218 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bafa26c-a31c-4a45-aea6-ed4f4a275e85" containerName="extract" Jul 16 10:37:00.716340 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.716223 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bafa26c-a31c-4a45-aea6-ed4f4a275e85" containerName="extract" Jul 16 10:37:00.716340 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.716272 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bafa26c-a31c-4a45-aea6-ed4f4a275e85" containerName="extract" Jul 16 10:37:00.719681 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.719666 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff" Jul 16 10:37:00.722774 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.722753 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 10:37:00.722980 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.722967 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 10:37:00.723905 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.723886 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4wnqd\"" Jul 16 10:37:00.729817 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.729786 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff"] Jul 16 10:37:00.758493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.755185 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53339cd6-473d-456d-a76a-cddd1884cf64-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fdhff\" (UID: \"53339cd6-473d-456d-a76a-cddd1884cf64\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff" Jul 16 10:37:00.758493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.755242 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97vsx\" (UniqueName: \"kubernetes.io/projected/53339cd6-473d-456d-a76a-cddd1884cf64-kube-api-access-97vsx\") pod \"openshift-lws-operator-fd8ccff4c-fdhff\" (UID: \"53339cd6-473d-456d-a76a-cddd1884cf64\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff" Jul 16 10:37:00.856131 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.856103 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53339cd6-473d-456d-a76a-cddd1884cf64-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fdhff\" (UID: \"53339cd6-473d-456d-a76a-cddd1884cf64\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff" Jul 16 10:37:00.856279 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.856140 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97vsx\" (UniqueName: \"kubernetes.io/projected/53339cd6-473d-456d-a76a-cddd1884cf64-kube-api-access-97vsx\") pod \"openshift-lws-operator-fd8ccff4c-fdhff\" (UID: \"53339cd6-473d-456d-a76a-cddd1884cf64\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff" Jul 16 10:37:00.856519 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.856500 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53339cd6-473d-456d-a76a-cddd1884cf64-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fdhff\" (UID: \"53339cd6-473d-456d-a76a-cddd1884cf64\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff" Jul 16 10:37:00.866087 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:00.866060 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97vsx\" (UniqueName: \"kubernetes.io/projected/53339cd6-473d-456d-a76a-cddd1884cf64-kube-api-access-97vsx\") pod \"openshift-lws-operator-fd8ccff4c-fdhff\" (UID: \"53339cd6-473d-456d-a76a-cddd1884cf64\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff" Jul 16 10:37:01.028559 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:01.028476 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff" Jul 16 10:37:01.149163 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:01.149141 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff"] Jul 16 10:37:01.151661 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:37:01.151634 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53339cd6_473d_456d_a76a_cddd1884cf64.slice/crio-1c40c08c6aa37a73e1f66bbf397b44aa30529fe8c9637fc0b14fa7735bec9163 WatchSource:0}: Error finding container 1c40c08c6aa37a73e1f66bbf397b44aa30529fe8c9637fc0b14fa7735bec9163: Status 404 returned error can't find the container with id 1c40c08c6aa37a73e1f66bbf397b44aa30529fe8c9637fc0b14fa7735bec9163 Jul 16 10:37:01.873453 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:01.873410 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff" event={"ID":"53339cd6-473d-456d-a76a-cddd1884cf64","Type":"ContainerStarted","Data":"1c40c08c6aa37a73e1f66bbf397b44aa30529fe8c9637fc0b14fa7735bec9163"} Jul 16 10:37:03.883045 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:03.883006 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff" event={"ID":"53339cd6-473d-456d-a76a-cddd1884cf64","Type":"ContainerStarted","Data":"7c1fe5edf09726c3f45548263a0a466095d78d43a6fcc8eeb2a843745bd883a5"} Jul 16 10:37:06.597989 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.597928 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fdhff" podStartSLOduration=4.878175474 podStartE2EDuration="6.597910963s" totalImagesPullingTime="1.719735489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:03.910229211 +0000 UTC m=+278.589494203" watchObservedRunningTime="2026-07-16 10:37:06.597910963 +0000 UTC m=+281.277175943" Jul 16 10:37:06.599270 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.599245 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6"] Jul 16 10:37:06.602729 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.602710 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:06.609349 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.609290 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 10:37:06.610374 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.610357 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 10:37:06.610461 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.610360 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9xvqh\"" Jul 16 10:37:06.615917 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.615894 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6"] Jul 16 10:37:06.696996 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.696969 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:06.697115 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.697003 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:06.697115 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.697032 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn854\" (UniqueName: \"kubernetes.io/projected/e6649748-6c04-4608-bd0e-bc93cff06bd6-kube-api-access-cn854\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:06.798133 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.798095 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:06.798133 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.798130 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:06.798353 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.798162 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn854\" (UniqueName: \"kubernetes.io/projected/e6649748-6c04-4608-bd0e-bc93cff06bd6-kube-api-access-cn854\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:06.798512 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.798490 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:06.798589 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.798519 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:06.807046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.807012 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn854\" (UniqueName: \"kubernetes.io/projected/e6649748-6c04-4608-bd0e-bc93cff06bd6-kube-api-access-cn854\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:06.911173 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:06.911087 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:07.035571 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:07.035452 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6"] Jul 16 10:37:07.038069 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:37:07.038038 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6649748_6c04_4608_bd0e_bc93cff06bd6.slice/crio-e1af72a765eabeca6b3291e86f0cbbab2f021b7dd2b807bfab761c8c15396740 WatchSource:0}: Error finding container e1af72a765eabeca6b3291e86f0cbbab2f021b7dd2b807bfab761c8c15396740: Status 404 returned error can't find the container with id e1af72a765eabeca6b3291e86f0cbbab2f021b7dd2b807bfab761c8c15396740 Jul 16 10:37:07.899588 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:07.899552 2581 generic.go:356] "Generic (PLEG): container finished" podID="e6649748-6c04-4608-bd0e-bc93cff06bd6" containerID="a56deab3e02603e750bc0e6847f8e4728931ea56061cf2a725257a0b0f6b8bad" exitCode=0 Jul 16 10:37:07.900063 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:07.899609 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" event={"ID":"e6649748-6c04-4608-bd0e-bc93cff06bd6","Type":"ContainerDied","Data":"a56deab3e02603e750bc0e6847f8e4728931ea56061cf2a725257a0b0f6b8bad"} Jul 16 10:37:07.900063 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:07.899646 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" event={"ID":"e6649748-6c04-4608-bd0e-bc93cff06bd6","Type":"ContainerStarted","Data":"e1af72a765eabeca6b3291e86f0cbbab2f021b7dd2b807bfab761c8c15396740"} Jul 16 10:37:08.904726 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:08.904697 2581 generic.go:356] "Generic (PLEG): container finished" podID="e6649748-6c04-4608-bd0e-bc93cff06bd6" containerID="b78e668c166500196b34667a574eb5d5d7f5648be86157df042dbc2c08b1e616" exitCode=0 Jul 16 10:37:08.905083 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:08.904784 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" event={"ID":"e6649748-6c04-4608-bd0e-bc93cff06bd6","Type":"ContainerDied","Data":"b78e668c166500196b34667a574eb5d5d7f5648be86157df042dbc2c08b1e616"} Jul 16 10:37:09.910763 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:09.910729 2581 generic.go:356] "Generic (PLEG): container finished" podID="e6649748-6c04-4608-bd0e-bc93cff06bd6" containerID="dca16c6ce5fc3fa10acf9fbb3e0b6afb130f87e318c8ca04187a327e06dc7692" exitCode=0 Jul 16 10:37:09.911184 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:09.910784 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" event={"ID":"e6649748-6c04-4608-bd0e-bc93cff06bd6","Type":"ContainerDied","Data":"dca16c6ce5fc3fa10acf9fbb3e0b6afb130f87e318c8ca04187a327e06dc7692"} Jul 16 10:37:11.062966 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.062933 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:11.133516 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.133481 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn854\" (UniqueName: \"kubernetes.io/projected/e6649748-6c04-4608-bd0e-bc93cff06bd6-kube-api-access-cn854\") pod \"e6649748-6c04-4608-bd0e-bc93cff06bd6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " Jul 16 10:37:11.133713 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.133551 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-bundle\") pod \"e6649748-6c04-4608-bd0e-bc93cff06bd6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " Jul 16 10:37:11.133713 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.133588 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-util\") pod \"e6649748-6c04-4608-bd0e-bc93cff06bd6\" (UID: \"e6649748-6c04-4608-bd0e-bc93cff06bd6\") " Jul 16 10:37:11.134267 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.134244 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-bundle" (OuterVolumeSpecName: "bundle") pod "e6649748-6c04-4608-bd0e-bc93cff06bd6" (UID: "e6649748-6c04-4608-bd0e-bc93cff06bd6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:37:11.135534 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.135511 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6649748-6c04-4608-bd0e-bc93cff06bd6-kube-api-access-cn854" (OuterVolumeSpecName: "kube-api-access-cn854") pod "e6649748-6c04-4608-bd0e-bc93cff06bd6" (UID: "e6649748-6c04-4608-bd0e-bc93cff06bd6"). InnerVolumeSpecName "kube-api-access-cn854". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:37:11.138937 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.138911 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-util" (OuterVolumeSpecName: "util") pod "e6649748-6c04-4608-bd0e-bc93cff06bd6" (UID: "e6649748-6c04-4608-bd0e-bc93cff06bd6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:37:11.234274 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.234177 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn854\" (UniqueName: \"kubernetes.io/projected/e6649748-6c04-4608-bd0e-bc93cff06bd6-kube-api-access-cn854\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:37:11.234274 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.234220 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-bundle\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:37:11.234274 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.234229 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6649748-6c04-4608-bd0e-bc93cff06bd6-util\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:37:11.920623 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.920591 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" event={"ID":"e6649748-6c04-4608-bd0e-bc93cff06bd6","Type":"ContainerDied","Data":"e1af72a765eabeca6b3291e86f0cbbab2f021b7dd2b807bfab761c8c15396740"} Jul 16 10:37:11.920800 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.920634 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1af72a765eabeca6b3291e86f0cbbab2f021b7dd2b807bfab761c8c15396740" Jul 16 10:37:11.920800 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:11.920635 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5z5b6" Jul 16 10:37:23.170304 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.170269 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt"] Jul 16 10:37:23.170731 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.170513 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6649748-6c04-4608-bd0e-bc93cff06bd6" containerName="pull" Jul 16 10:37:23.170731 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.170524 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6649748-6c04-4608-bd0e-bc93cff06bd6" containerName="pull" Jul 16 10:37:23.170731 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.170533 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6649748-6c04-4608-bd0e-bc93cff06bd6" containerName="extract" Jul 16 10:37:23.170731 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.170539 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6649748-6c04-4608-bd0e-bc93cff06bd6" containerName="extract" Jul 16 10:37:23.170731 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.170555 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6649748-6c04-4608-bd0e-bc93cff06bd6" containerName="util" Jul 16 10:37:23.170731 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.170561 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6649748-6c04-4608-bd0e-bc93cff06bd6" containerName="util" Jul 16 10:37:23.170731 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.170614 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6649748-6c04-4608-bd0e-bc93cff06bd6" containerName="extract" Jul 16 10:37:23.174474 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.174456 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.179258 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.179235 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 10:37:23.181864 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.181845 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 10:37:23.181971 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.181908 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 10:37:23.182046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.181995 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-j9zwf\"" Jul 16 10:37:23.182471 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.182454 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 10:37:23.221708 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.221659 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-tmp\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.221883 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.221743 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.221883 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.221808 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdj47\" (UniqueName: \"kubernetes.io/projected/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-kube-api-access-vdj47\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.221996 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.221875 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-manifests\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.221996 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.221926 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.223081 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.223049 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt"] Jul 16 10:37:23.322987 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.322952 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdj47\" (UniqueName: \"kubernetes.io/projected/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-kube-api-access-vdj47\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.322987 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.322992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-manifests\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.323183 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.323013 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.323183 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.323062 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-tmp\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.323183 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.323086 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.323410 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.323378 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-manifests\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.325488 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.325457 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-tmp\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.325609 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.325546 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.325682 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.325625 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-webhook-cert\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.356086 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.356050 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdj47\" (UniqueName: \"kubernetes.io/projected/eb815613-68ff-455c-9bf6-e69e1f6a8a8b-kube-api-access-vdj47\") pod \"opendatahub-operator-controller-manager-d8f7d9f46-tntpt\" (UID: \"eb815613-68ff-455c-9bf6-e69e1f6a8a8b\") " pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.484561 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.484466 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:23.629498 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.629475 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt"] Jul 16 10:37:23.631920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:37:23.631893 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb815613_68ff_455c_9bf6_e69e1f6a8a8b.slice/crio-6ed91baddccc10fdca0ec87761e10b998014f8965b3233046ee68f889a14a0a2 WatchSource:0}: Error finding container 6ed91baddccc10fdca0ec87761e10b998014f8965b3233046ee68f889a14a0a2: Status 404 returned error can't find the container with id 6ed91baddccc10fdca0ec87761e10b998014f8965b3233046ee68f889a14a0a2 Jul 16 10:37:23.970776 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:23.970744 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" event={"ID":"eb815613-68ff-455c-9bf6-e69e1f6a8a8b","Type":"ContainerStarted","Data":"6ed91baddccc10fdca0ec87761e10b998014f8965b3233046ee68f889a14a0a2"} Jul 16 10:37:26.554596 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:26.554555 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:37:26.557082 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:26.554759 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:37:26.557363 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:26.557348 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 10:37:26.986052 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:26.986010 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" event={"ID":"eb815613-68ff-455c-9bf6-e69e1f6a8a8b","Type":"ContainerStarted","Data":"9c286718d9bdb634479a711f47e0987c75a1f2fec9b8924a9b391da55e517ada"} Jul 16 10:37:27.991278 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:27.991183 2581 generic.go:356] "Generic (PLEG): container finished" podID="eb815613-68ff-455c-9bf6-e69e1f6a8a8b" containerID="9c286718d9bdb634479a711f47e0987c75a1f2fec9b8924a9b391da55e517ada" exitCode=0 Jul 16 10:37:27.991626 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:27.991271 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" event={"ID":"eb815613-68ff-455c-9bf6-e69e1f6a8a8b","Type":"ContainerDied","Data":"9c286718d9bdb634479a711f47e0987c75a1f2fec9b8924a9b391da55e517ada"} Jul 16 10:37:27.991758 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:27.991743 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 10:37:28.997588 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:28.997556 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" event={"ID":"eb815613-68ff-455c-9bf6-e69e1f6a8a8b","Type":"ContainerStarted","Data":"1a5404fa4680ae9fd88987e450dc432d6f6d0cbd0d868c4d265604c35e644e8d"} Jul 16 10:37:28.997967 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:28.997703 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:29.025313 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:29.025265 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" podStartSLOduration=1.540749653 podStartE2EDuration="6.025250388s" totalImagesPullingTime="3.387178723s" totalInitContainerRuntime="1.097322012s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:29.024561256 +0000 UTC m=+303.703826252" watchObservedRunningTime="2026-07-16 10:37:29.025250388 +0000 UTC m=+303.704515365" Jul 16 10:37:40.004281 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:40.004254 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d8f7d9f46-tntpt" Jul 16 10:37:45.527667 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.527630 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b"] Jul 16 10:37:45.536334 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.536305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:45.543409 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:37:45.543312 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-creds\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" type="*v1.Secret" Jul 16 10:37:45.544254 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.544228 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 10:37:45.544367 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.544285 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 16 10:37:45.544777 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.544761 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 10:37:45.576949 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.576913 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b"] Jul 16 10:37:45.603707 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.603656 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e635e43-7c5c-403a-be45-836800d9fb89-tmp\") pod \"kube-auth-proxy-58946d4b5c-wcc4b\" (UID: \"8e635e43-7c5c-403a-be45-836800d9fb89\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:45.603904 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.603728 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj8qb\" (UniqueName: \"kubernetes.io/projected/8e635e43-7c5c-403a-be45-836800d9fb89-kube-api-access-cj8qb\") pod \"kube-auth-proxy-58946d4b5c-wcc4b\" (UID: \"8e635e43-7c5c-403a-be45-836800d9fb89\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:45.603904 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.603851 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e635e43-7c5c-403a-be45-836800d9fb89-tls-certs\") pod \"kube-auth-proxy-58946d4b5c-wcc4b\" (UID: \"8e635e43-7c5c-403a-be45-836800d9fb89\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:45.649137 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.649096 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d9mmp"] Jul 16 10:37:45.652636 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.652611 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" Jul 16 10:37:45.659410 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.659270 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 16 10:37:45.660585 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.660560 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-pblbp\"" Jul 16 10:37:45.666960 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.666913 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d9mmp"] Jul 16 10:37:45.704708 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.704665 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e635e43-7c5c-403a-be45-836800d9fb89-tls-certs\") pod \"kube-auth-proxy-58946d4b5c-wcc4b\" (UID: \"8e635e43-7c5c-403a-be45-836800d9fb89\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:45.704899 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.704747 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e635e43-7c5c-403a-be45-836800d9fb89-tmp\") pod \"kube-auth-proxy-58946d4b5c-wcc4b\" (UID: \"8e635e43-7c5c-403a-be45-836800d9fb89\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:45.704899 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.704775 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj8qb\" (UniqueName: \"kubernetes.io/projected/8e635e43-7c5c-403a-be45-836800d9fb89-kube-api-access-cj8qb\") pod \"kube-auth-proxy-58946d4b5c-wcc4b\" (UID: \"8e635e43-7c5c-403a-be45-836800d9fb89\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:45.704899 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:37:45.704794 2581 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 16 10:37:45.704899 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:37:45.704856 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e635e43-7c5c-403a-be45-836800d9fb89-tls-certs podName:8e635e43-7c5c-403a-be45-836800d9fb89 nodeName:}" failed. No retries permitted until 2026-07-16 10:37:46.204839414 +0000 UTC m=+320.884104371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8e635e43-7c5c-403a-be45-836800d9fb89-tls-certs") pod "kube-auth-proxy-58946d4b5c-wcc4b" (UID: "8e635e43-7c5c-403a-be45-836800d9fb89") : secret "kube-auth-proxy-tls" not found Jul 16 10:37:45.707044 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.707024 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e635e43-7c5c-403a-be45-836800d9fb89-tmp\") pod \"kube-auth-proxy-58946d4b5c-wcc4b\" (UID: \"8e635e43-7c5c-403a-be45-836800d9fb89\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:45.720033 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.720006 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj8qb\" (UniqueName: \"kubernetes.io/projected/8e635e43-7c5c-403a-be45-836800d9fb89-kube-api-access-cj8qb\") pod \"kube-auth-proxy-58946d4b5c-wcc4b\" (UID: \"8e635e43-7c5c-403a-be45-836800d9fb89\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:45.805282 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.805192 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08dc4380-9e09-4e3a-ab75-dd064a191ebe-cert\") pod \"odh-model-controller-7998c67dc4-d9mmp\" (UID: \"08dc4380-9e09-4e3a-ab75-dd064a191ebe\") " pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" Jul 16 10:37:45.805282 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.805245 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7wdb\" (UniqueName: \"kubernetes.io/projected/08dc4380-9e09-4e3a-ab75-dd064a191ebe-kube-api-access-c7wdb\") pod \"odh-model-controller-7998c67dc4-d9mmp\" (UID: \"08dc4380-9e09-4e3a-ab75-dd064a191ebe\") " pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" Jul 16 10:37:45.905857 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.905823 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08dc4380-9e09-4e3a-ab75-dd064a191ebe-cert\") pod \"odh-model-controller-7998c67dc4-d9mmp\" (UID: \"08dc4380-9e09-4e3a-ab75-dd064a191ebe\") " pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" Jul 16 10:37:45.906062 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.905880 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7wdb\" (UniqueName: \"kubernetes.io/projected/08dc4380-9e09-4e3a-ab75-dd064a191ebe-kube-api-access-c7wdb\") pod \"odh-model-controller-7998c67dc4-d9mmp\" (UID: \"08dc4380-9e09-4e3a-ab75-dd064a191ebe\") " pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" Jul 16 10:37:45.908313 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.908287 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08dc4380-9e09-4e3a-ab75-dd064a191ebe-cert\") pod \"odh-model-controller-7998c67dc4-d9mmp\" (UID: \"08dc4380-9e09-4e3a-ab75-dd064a191ebe\") " pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" Jul 16 10:37:45.931545 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.931507 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7wdb\" (UniqueName: \"kubernetes.io/projected/08dc4380-9e09-4e3a-ab75-dd064a191ebe-kube-api-access-c7wdb\") pod \"odh-model-controller-7998c67dc4-d9mmp\" (UID: \"08dc4380-9e09-4e3a-ab75-dd064a191ebe\") " pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" Jul 16 10:37:45.963519 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:45.963486 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" Jul 16 10:37:46.130440 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.130400 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-d9mmp"] Jul 16 10:37:46.134011 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:37:46.133985 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08dc4380_9e09_4e3a_ab75_dd064a191ebe.slice/crio-4b5a684634a29f25a6098fe481708fc378705314e3282ce4ff2a21957bbce424 WatchSource:0}: Error finding container 4b5a684634a29f25a6098fe481708fc378705314e3282ce4ff2a21957bbce424: Status 404 returned error can't find the container with id 4b5a684634a29f25a6098fe481708fc378705314e3282ce4ff2a21957bbce424 Jul 16 10:37:46.208713 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.208654 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e635e43-7c5c-403a-be45-836800d9fb89-tls-certs\") pod \"kube-auth-proxy-58946d4b5c-wcc4b\" (UID: \"8e635e43-7c5c-403a-be45-836800d9fb89\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:46.211036 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.211014 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e635e43-7c5c-403a-be45-836800d9fb89-tls-certs\") pod \"kube-auth-proxy-58946d4b5c-wcc4b\" (UID: \"8e635e43-7c5c-403a-be45-836800d9fb89\") " pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:46.446150 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.446058 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" Jul 16 10:37:46.603513 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.603459 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b"] Jul 16 10:37:46.605974 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:37:46.605948 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e635e43_7c5c_403a_be45_836800d9fb89.slice/crio-296f1cc132a96b93c6e4e74792840e082875005d628a685226bf0db97c306772 WatchSource:0}: Error finding container 296f1cc132a96b93c6e4e74792840e082875005d628a685226bf0db97c306772: Status 404 returned error can't find the container with id 296f1cc132a96b93c6e4e74792840e082875005d628a685226bf0db97c306772 Jul 16 10:37:46.654983 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.654950 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5q7wq"] Jul 16 10:37:46.659160 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.659139 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" Jul 16 10:37:46.663401 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.663381 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-kghmt\"" Jul 16 10:37:46.663895 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.663879 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 16 10:37:46.693447 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.693405 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5q7wq"] Jul 16 10:37:46.814450 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.814407 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99c1a236-aca2-4120-9a1d-473b2a4ef53e-tls-certs\") pod \"model-serving-api-54d89bd5fc-5q7wq\" (UID: \"99c1a236-aca2-4120-9a1d-473b2a4ef53e\") " pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" Jul 16 10:37:46.814637 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.814482 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftdtt\" (UniqueName: \"kubernetes.io/projected/99c1a236-aca2-4120-9a1d-473b2a4ef53e-kube-api-access-ftdtt\") pod \"model-serving-api-54d89bd5fc-5q7wq\" (UID: \"99c1a236-aca2-4120-9a1d-473b2a4ef53e\") " pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" Jul 16 10:37:46.819129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.819107 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 16 10:37:46.916073 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.915603 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99c1a236-aca2-4120-9a1d-473b2a4ef53e-tls-certs\") pod \"model-serving-api-54d89bd5fc-5q7wq\" (UID: \"99c1a236-aca2-4120-9a1d-473b2a4ef53e\") " pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" Jul 16 10:37:46.916073 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:37:46.915765 2581 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 16 10:37:46.916073 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:37:46.915846 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99c1a236-aca2-4120-9a1d-473b2a4ef53e-tls-certs podName:99c1a236-aca2-4120-9a1d-473b2a4ef53e nodeName:}" failed. No retries permitted until 2026-07-16 10:37:47.415818127 +0000 UTC m=+322.095083098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/99c1a236-aca2-4120-9a1d-473b2a4ef53e-tls-certs") pod "model-serving-api-54d89bd5fc-5q7wq" (UID: "99c1a236-aca2-4120-9a1d-473b2a4ef53e") : secret "model-serving-api-tls" not found Jul 16 10:37:46.916073 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.916017 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftdtt\" (UniqueName: \"kubernetes.io/projected/99c1a236-aca2-4120-9a1d-473b2a4ef53e-kube-api-access-ftdtt\") pod \"model-serving-api-54d89bd5fc-5q7wq\" (UID: \"99c1a236-aca2-4120-9a1d-473b2a4ef53e\") " pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" Jul 16 10:37:46.942964 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:46.942927 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftdtt\" (UniqueName: \"kubernetes.io/projected/99c1a236-aca2-4120-9a1d-473b2a4ef53e-kube-api-access-ftdtt\") pod \"model-serving-api-54d89bd5fc-5q7wq\" (UID: \"99c1a236-aca2-4120-9a1d-473b2a4ef53e\") " pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" Jul 16 10:37:47.073362 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.073261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" event={"ID":"8e635e43-7c5c-403a-be45-836800d9fb89","Type":"ContainerStarted","Data":"296f1cc132a96b93c6e4e74792840e082875005d628a685226bf0db97c306772"} Jul 16 10:37:47.074826 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.074793 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" event={"ID":"08dc4380-9e09-4e3a-ab75-dd064a191ebe","Type":"ContainerStarted","Data":"4b5a684634a29f25a6098fe481708fc378705314e3282ce4ff2a21957bbce424"} Jul 16 10:37:47.420580 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.420427 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99c1a236-aca2-4120-9a1d-473b2a4ef53e-tls-certs\") pod \"model-serving-api-54d89bd5fc-5q7wq\" (UID: \"99c1a236-aca2-4120-9a1d-473b2a4ef53e\") " pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" Jul 16 10:37:47.430616 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.430581 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99c1a236-aca2-4120-9a1d-473b2a4ef53e-tls-certs\") pod \"model-serving-api-54d89bd5fc-5q7wq\" (UID: \"99c1a236-aca2-4120-9a1d-473b2a4ef53e\") " pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" Jul 16 10:37:47.572108 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.572072 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" Jul 16 10:37:47.613323 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.613111 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2"] Jul 16 10:37:47.616520 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.616491 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.642805 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.641889 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 10:37:47.643868 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.643178 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 10:37:47.643868 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.643404 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 10:37:47.643868 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.643629 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ght6v\"" Jul 16 10:37:47.648508 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.648363 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 10:37:47.723752 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.723601 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2"] Jul 16 10:37:47.724433 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.723970 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.724433 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.724050 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-token\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.724433 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.724087 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-cacerts\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.724433 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.724131 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.724433 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.724179 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b2653ef-275c-43bb-8275-6b11b68ef18e-local-certs\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.724433 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.724209 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqrkj\" (UniqueName: \"kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-kube-api-access-qqrkj\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.724433 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.724269 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.784313 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.784282 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5q7wq"] Jul 16 10:37:47.825489 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.825448 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.825654 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.825564 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b2653ef-275c-43bb-8275-6b11b68ef18e-local-certs\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.825654 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.825591 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrkj\" (UniqueName: \"kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-kube-api-access-qqrkj\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.825654 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.825630 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.825854 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.825723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.825854 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.825753 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-token\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.825854 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.825783 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-cacerts\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.826552 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.826480 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.828969 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.828947 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-cacerts\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.829192 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.829140 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.830285 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.829974 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.830834 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.830812 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b2653ef-275c-43bb-8275-6b11b68ef18e-local-certs\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.853973 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.853944 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-token\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.876949 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.876916 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqrkj\" (UniqueName: \"kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-kube-api-access-qqrkj\") pod \"istiod-openshift-gateway-7574459656-qd4s2\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:47.930227 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:47.930163 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:48.082369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:48.082318 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" event={"ID":"99c1a236-aca2-4120-9a1d-473b2a4ef53e","Type":"ContainerStarted","Data":"5c41ec40766ad92953897ee8a9d3430d6b6bd7c175f602854054c4bbce1bf6f5"} Jul 16 10:37:50.201940 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:50.201892 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2"] Jul 16 10:37:50.204188 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:37:50.204160 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2653ef_275c_43bb_8275_6b11b68ef18e.slice/crio-b67d7fde50b98f88306fde64d3ade43660a6fd083df75cafa7bf8775b115b42f WatchSource:0}: Error finding container b67d7fde50b98f88306fde64d3ade43660a6fd083df75cafa7bf8775b115b42f: Status 404 returned error can't find the container with id b67d7fde50b98f88306fde64d3ade43660a6fd083df75cafa7bf8775b115b42f Jul 16 10:37:51.101423 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:51.101376 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" event={"ID":"5b2653ef-275c-43bb-8275-6b11b68ef18e","Type":"ContainerStarted","Data":"b67d7fde50b98f88306fde64d3ade43660a6fd083df75cafa7bf8775b115b42f"} Jul 16 10:37:51.103398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:51.103361 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" event={"ID":"08dc4380-9e09-4e3a-ab75-dd064a191ebe","Type":"ContainerStarted","Data":"c768fab34fac13cd40c8fa4f1b984ab2eb03c1ee51caedcfdf4eed4328bb5291"} Jul 16 10:37:51.103747 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:51.103725 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" Jul 16 10:37:51.105826 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:51.105798 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" event={"ID":"8e635e43-7c5c-403a-be45-836800d9fb89","Type":"ContainerStarted","Data":"0bcb3db4975ae3a7391e0fea27e2227b724ff40ddde6ea2e3259e118a1b5d106"} Jul 16 10:37:51.108002 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:51.107550 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" event={"ID":"99c1a236-aca2-4120-9a1d-473b2a4ef53e","Type":"ContainerStarted","Data":"442533147c95c00da4f721e991d8aec3a94b9da41d441beed391f9faf436ea87"} Jul 16 10:37:51.108002 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:51.107751 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" Jul 16 10:37:51.154427 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:51.154261 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" podStartSLOduration=2.2600368 podStartE2EDuration="6.154240021s" totalImagesPullingTime="3.894203221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:51.148583602 +0000 UTC m=+325.827848583" watchObservedRunningTime="2026-07-16 10:37:51.154240021 +0000 UTC m=+325.833505002" Jul 16 10:37:51.217558 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:51.217505 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58946d4b5c-wcc4b" podStartSLOduration=2.801208834 podStartE2EDuration="6.217487697s" totalImagesPullingTime="3.416278863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:51.216460865 +0000 UTC m=+325.895725857" watchObservedRunningTime="2026-07-16 10:37:51.217487697 +0000 UTC m=+325.896752677" Jul 16 10:37:52.716794 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:52.716745 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 10:37:52.717120 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:52.716840 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 10:37:53.119494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:53.119449 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" event={"ID":"5b2653ef-275c-43bb-8275-6b11b68ef18e","Type":"ContainerStarted","Data":"0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d"} Jul 16 10:37:53.119655 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:53.119600 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:37:53.121323 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:53.121294 2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-qd4s2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 10:37:53.121442 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:53.121380 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" podUID="5b2653ef-275c-43bb-8275-6b11b68ef18e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 10:37:53.159130 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:53.159079 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" podStartSLOduration=4.283660351 podStartE2EDuration="7.159061921s" totalImagesPullingTime="2.87540157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:51.264295152 +0000 UTC m=+325.943560133" watchObservedRunningTime="2026-07-16 10:37:53.159061921 +0000 UTC m=+327.838326944" Jul 16 10:37:53.159353 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:53.159333 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" podStartSLOduration=3.649306781 podStartE2EDuration="6.159328774s" totalImagesPullingTime="2.510021993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:37:53.156091189 +0000 UTC m=+327.835356169" watchObservedRunningTime="2026-07-16 10:37:53.159328774 +0000 UTC m=+327.838593754" Jul 16 10:37:54.124185 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:54.124152 2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-qd4s2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 10:37:54.124545 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:54.124215 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" podUID="5b2653ef-275c-43bb-8275-6b11b68ef18e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 10:37:57.124310 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:37:57.124272 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:38:02.115106 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:02.115076 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-d9mmp" Jul 16 10:38:02.116899 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:02.116881 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-5q7wq" Jul 16 10:38:03.265615 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.265577 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c"] Jul 16 10:38:03.275885 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.275858 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" Jul 16 10:38:03.306981 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.306954 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 16 10:38:03.308511 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.308495 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bv2c9\"" Jul 16 10:38:03.352240 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.352208 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/536d56ea-4060-4215-b36b-9b7e2373466c-cert\") pod \"kserve-controller-manager-d6f4b65b8-7rw8c\" (UID: \"536d56ea-4060-4215-b36b-9b7e2373466c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" Jul 16 10:38:03.352421 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.352293 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7s64\" (UniqueName: \"kubernetes.io/projected/536d56ea-4060-4215-b36b-9b7e2373466c-kube-api-access-d7s64\") pod \"kserve-controller-manager-d6f4b65b8-7rw8c\" (UID: \"536d56ea-4060-4215-b36b-9b7e2373466c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" Jul 16 10:38:03.385051 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.385019 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c"] Jul 16 10:38:03.432574 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.432540 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm"] Jul 16 10:38:03.435732 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.435714 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" Jul 16 10:38:03.439743 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.439674 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-mjbv6\"" Jul 16 10:38:03.440562 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.440539 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 16 10:38:03.453467 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.453441 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/536d56ea-4060-4215-b36b-9b7e2373466c-cert\") pod \"kserve-controller-manager-d6f4b65b8-7rw8c\" (UID: \"536d56ea-4060-4215-b36b-9b7e2373466c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" Jul 16 10:38:03.453593 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.453474 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch5sq\" (UniqueName: \"kubernetes.io/projected/8632f28b-52ee-4b02-9106-1bc9363c4387-kube-api-access-ch5sq\") pod \"llmisvc-controller-manager-777c6bd7d5-tkmkm\" (UID: \"8632f28b-52ee-4b02-9106-1bc9363c4387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" Jul 16 10:38:03.453593 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.453520 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8632f28b-52ee-4b02-9106-1bc9363c4387-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tkmkm\" (UID: \"8632f28b-52ee-4b02-9106-1bc9363c4387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" Jul 16 10:38:03.453593 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.453541 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7s64\" (UniqueName: \"kubernetes.io/projected/536d56ea-4060-4215-b36b-9b7e2373466c-kube-api-access-d7s64\") pod \"kserve-controller-manager-d6f4b65b8-7rw8c\" (UID: \"536d56ea-4060-4215-b36b-9b7e2373466c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" Jul 16 10:38:03.455819 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.455800 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/536d56ea-4060-4215-b36b-9b7e2373466c-cert\") pod \"kserve-controller-manager-d6f4b65b8-7rw8c\" (UID: \"536d56ea-4060-4215-b36b-9b7e2373466c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" Jul 16 10:38:03.467210 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.467187 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm"] Jul 16 10:38:03.495287 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.495247 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7s64\" (UniqueName: \"kubernetes.io/projected/536d56ea-4060-4215-b36b-9b7e2373466c-kube-api-access-d7s64\") pod \"kserve-controller-manager-d6f4b65b8-7rw8c\" (UID: \"536d56ea-4060-4215-b36b-9b7e2373466c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" Jul 16 10:38:03.554267 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.554182 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8632f28b-52ee-4b02-9106-1bc9363c4387-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tkmkm\" (UID: \"8632f28b-52ee-4b02-9106-1bc9363c4387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" Jul 16 10:38:03.554267 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.554247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch5sq\" (UniqueName: \"kubernetes.io/projected/8632f28b-52ee-4b02-9106-1bc9363c4387-kube-api-access-ch5sq\") pod \"llmisvc-controller-manager-777c6bd7d5-tkmkm\" (UID: \"8632f28b-52ee-4b02-9106-1bc9363c4387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" Jul 16 10:38:03.556641 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.556617 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8632f28b-52ee-4b02-9106-1bc9363c4387-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tkmkm\" (UID: \"8632f28b-52ee-4b02-9106-1bc9363c4387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" Jul 16 10:38:03.569975 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.569951 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch5sq\" (UniqueName: \"kubernetes.io/projected/8632f28b-52ee-4b02-9106-1bc9363c4387-kube-api-access-ch5sq\") pod \"llmisvc-controller-manager-777c6bd7d5-tkmkm\" (UID: \"8632f28b-52ee-4b02-9106-1bc9363c4387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" Jul 16 10:38:03.585821 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.585793 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" Jul 16 10:38:03.730083 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.730060 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c"] Jul 16 10:38:03.731628 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:38:03.731602 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536d56ea_4060_4215_b36b_9b7e2373466c.slice/crio-2738f1b17ff5d4668cdb63a061f4e172dfd1ca46eff7c24fae635ded4e550247 WatchSource:0}: Error finding container 2738f1b17ff5d4668cdb63a061f4e172dfd1ca46eff7c24fae635ded4e550247: Status 404 returned error can't find the container with id 2738f1b17ff5d4668cdb63a061f4e172dfd1ca46eff7c24fae635ded4e550247 Jul 16 10:38:03.745521 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.745497 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" Jul 16 10:38:03.971066 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:03.971043 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm"] Jul 16 10:38:03.972521 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:38:03.972494 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8632f28b_52ee_4b02_9106_1bc9363c4387.slice/crio-c68906e71b2b70e7a63fd00720483b9070e207731a8461bdf8d03d5ca6d303b3 WatchSource:0}: Error finding container c68906e71b2b70e7a63fd00720483b9070e207731a8461bdf8d03d5ca6d303b3: Status 404 returned error can't find the container with id c68906e71b2b70e7a63fd00720483b9070e207731a8461bdf8d03d5ca6d303b3 Jul 16 10:38:04.169795 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.169678 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" event={"ID":"8632f28b-52ee-4b02-9106-1bc9363c4387","Type":"ContainerStarted","Data":"c68906e71b2b70e7a63fd00720483b9070e207731a8461bdf8d03d5ca6d303b3"} Jul 16 10:38:04.170888 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.170867 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" event={"ID":"536d56ea-4060-4215-b36b-9b7e2373466c","Type":"ContainerStarted","Data":"2738f1b17ff5d4668cdb63a061f4e172dfd1ca46eff7c24fae635ded4e550247"} Jul 16 10:38:04.447971 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.447891 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2"] Jul 16 10:38:04.448332 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.448081 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" podUID="5b2653ef-275c-43bb-8275-6b11b68ef18e" containerName="discovery" containerID="cri-o://0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d" gracePeriod=30 Jul 16 10:38:04.727524 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.727493 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:38:04.765227 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.765191 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-dns-cert\") pod \"5b2653ef-275c-43bb-8275-6b11b68ef18e\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " Jul 16 10:38:04.765410 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.765237 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-token\") pod \"5b2653ef-275c-43bb-8275-6b11b68ef18e\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " Jul 16 10:38:04.765410 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.765259 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-cacerts\") pod \"5b2653ef-275c-43bb-8275-6b11b68ef18e\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " Jul 16 10:38:04.765410 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.765331 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqrkj\" (UniqueName: \"kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-kube-api-access-qqrkj\") pod \"5b2653ef-275c-43bb-8275-6b11b68ef18e\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " Jul 16 10:38:04.765410 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.765379 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-ca-configmap\") pod \"5b2653ef-275c-43bb-8275-6b11b68ef18e\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " Jul 16 10:38:04.765410 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.765410 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-kubeconfig\") pod \"5b2653ef-275c-43bb-8275-6b11b68ef18e\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " Jul 16 10:38:04.765750 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.765438 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b2653ef-275c-43bb-8275-6b11b68ef18e-local-certs\") pod \"5b2653ef-275c-43bb-8275-6b11b68ef18e\" (UID: \"5b2653ef-275c-43bb-8275-6b11b68ef18e\") " Jul 16 10:38:04.766371 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.766325 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "5b2653ef-275c-43bb-8275-6b11b68ef18e" (UID: "5b2653ef-275c-43bb-8275-6b11b68ef18e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:38:04.770020 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.769934 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-kube-api-access-qqrkj" (OuterVolumeSpecName: "kube-api-access-qqrkj") pod "5b2653ef-275c-43bb-8275-6b11b68ef18e" (UID: "5b2653ef-275c-43bb-8275-6b11b68ef18e"). InnerVolumeSpecName "kube-api-access-qqrkj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:38:04.772294 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.771505 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b2653ef-275c-43bb-8275-6b11b68ef18e-local-certs" (OuterVolumeSpecName: "local-certs") pod "5b2653ef-275c-43bb-8275-6b11b68ef18e" (UID: "5b2653ef-275c-43bb-8275-6b11b68ef18e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:38:04.772294 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.771462 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-token" (OuterVolumeSpecName: "istio-token") pod "5b2653ef-275c-43bb-8275-6b11b68ef18e" (UID: "5b2653ef-275c-43bb-8275-6b11b68ef18e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:38:04.772865 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.772661 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "5b2653ef-275c-43bb-8275-6b11b68ef18e" (UID: "5b2653ef-275c-43bb-8275-6b11b68ef18e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:38:04.772865 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.772838 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-cacerts" (OuterVolumeSpecName: "cacerts") pod "5b2653ef-275c-43bb-8275-6b11b68ef18e" (UID: "5b2653ef-275c-43bb-8275-6b11b68ef18e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:38:04.774254 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.774180 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "5b2653ef-275c-43bb-8275-6b11b68ef18e" (UID: "5b2653ef-275c-43bb-8275-6b11b68ef18e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:38:04.866274 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.866238 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-dns-cert\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:38:04.866274 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.866270 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-token\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:38:04.866274 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.866280 2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-cacerts\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:38:04.866274 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.866288 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqrkj\" (UniqueName: \"kubernetes.io/projected/5b2653ef-275c-43bb-8275-6b11b68ef18e-kube-api-access-qqrkj\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:38:04.866568 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.866297 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-csr-ca-configmap\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:38:04.866568 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.866306 2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5b2653ef-275c-43bb-8275-6b11b68ef18e-istio-kubeconfig\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:38:04.866568 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:04.866314 2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b2653ef-275c-43bb-8275-6b11b68ef18e-local-certs\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:38:05.181721 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:05.181669 2581 generic.go:356] "Generic (PLEG): container finished" podID="5b2653ef-275c-43bb-8275-6b11b68ef18e" containerID="0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d" exitCode=0 Jul 16 10:38:05.181886 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:05.181743 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" event={"ID":"5b2653ef-275c-43bb-8275-6b11b68ef18e","Type":"ContainerDied","Data":"0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d"} Jul 16 10:38:05.181886 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:05.181753 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" Jul 16 10:38:05.181886 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:05.181813 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2" event={"ID":"5b2653ef-275c-43bb-8275-6b11b68ef18e","Type":"ContainerDied","Data":"b67d7fde50b98f88306fde64d3ade43660a6fd083df75cafa7bf8775b115b42f"} Jul 16 10:38:05.181886 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:05.181836 2581 scope.go:117] "RemoveContainer" containerID="0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d" Jul 16 10:38:05.225375 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:05.225330 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2"] Jul 16 10:38:05.247425 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:05.247374 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qd4s2"] Jul 16 10:38:05.307123 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:05.307101 2581 scope.go:117] "RemoveContainer" containerID="0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d" Jul 16 10:38:05.307507 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:38:05.307483 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d\": container with ID starting with 0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d not found: ID does not exist" containerID="0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d" Jul 16 10:38:05.307595 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:05.307515 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d"} err="failed to get container status \"0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d\": rpc error: code = NotFound desc = could not find container \"0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d\": container with ID starting with 0534f7810b9829159f134322f947bbe0e432311d561037fc781e6c2c9560558d not found: ID does not exist" Jul 16 10:38:05.807241 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:05.807200 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b2653ef-275c-43bb-8275-6b11b68ef18e" path="/var/lib/kubelet/pods/5b2653ef-275c-43bb-8275-6b11b68ef18e/volumes" Jul 16 10:38:07.194987 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:07.194951 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" event={"ID":"8632f28b-52ee-4b02-9106-1bc9363c4387","Type":"ContainerStarted","Data":"641169c3ccf306be2cda165ff308a5963a82b494467e95021164d9eb98d19584"} Jul 16 10:38:07.195407 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:07.195038 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" Jul 16 10:38:07.196564 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:07.196537 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" event={"ID":"536d56ea-4060-4215-b36b-9b7e2373466c","Type":"ContainerStarted","Data":"ea0e4a05c432371f63dc2cd8e1fece7c33dc98d4baef66faa4a3585b15e6bf98"} Jul 16 10:38:07.196723 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:07.196605 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" Jul 16 10:38:07.284443 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:07.284323 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" podStartSLOduration=1.27397097 podStartE2EDuration="4.284300002s" totalImagesPullingTime="3.010329032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:38:07.234357926 +0000 UTC m=+341.913622906" watchObservedRunningTime="2026-07-16 10:38:07.284300002 +0000 UTC m=+341.963564982" Jul 16 10:38:07.285863 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:07.285828 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" podStartSLOduration=1.849333745 podStartE2EDuration="4.285817084s" totalImagesPullingTime="2.436483339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:38:07.284256895 +0000 UTC m=+341.963521875" watchObservedRunningTime="2026-07-16 10:38:07.285817084 +0000 UTC m=+341.965082084" Jul 16 10:38:38.203833 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:38.203804 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkmkm" Jul 16 10:38:38.206906 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:38.206885 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7rw8c" Jul 16 10:38:45.837974 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:45.837943 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qwmgq"] Jul 16 10:38:45.838333 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:45.838239 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2653ef-275c-43bb-8275-6b11b68ef18e" containerName="discovery" Jul 16 10:38:45.838333 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:45.838251 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2653ef-275c-43bb-8275-6b11b68ef18e" containerName="discovery" Jul 16 10:38:45.838333 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:45.838315 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2653ef-275c-43bb-8275-6b11b68ef18e" containerName="discovery" Jul 16 10:38:45.842826 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:45.842805 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" Jul 16 10:38:45.845941 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:45.845918 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-j9c4l\"" Jul 16 10:38:45.846068 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:45.845955 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 10:38:45.846068 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:45.845986 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 10:38:45.851612 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:45.851586 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qwmgq"] Jul 16 10:38:46.006221 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.006188 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs96r\" (UniqueName: \"kubernetes.io/projected/99c922ca-2a7f-4375-9f3a-28c033fecb1c-kube-api-access-fs96r\") pod \"kuadrant-operator-catalog-qwmgq\" (UID: \"99c922ca-2a7f-4375-9f3a-28c033fecb1c\") " pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" Jul 16 10:38:46.107616 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.107522 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs96r\" (UniqueName: \"kubernetes.io/projected/99c922ca-2a7f-4375-9f3a-28c033fecb1c-kube-api-access-fs96r\") pod \"kuadrant-operator-catalog-qwmgq\" (UID: \"99c922ca-2a7f-4375-9f3a-28c033fecb1c\") " pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" Jul 16 10:38:46.119741 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.119713 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs96r\" (UniqueName: \"kubernetes.io/projected/99c922ca-2a7f-4375-9f3a-28c033fecb1c-kube-api-access-fs96r\") pod \"kuadrant-operator-catalog-qwmgq\" (UID: \"99c922ca-2a7f-4375-9f3a-28c033fecb1c\") " pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" Jul 16 10:38:46.152777 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.152750 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" Jul 16 10:38:46.191464 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.191429 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qwmgq"] Jul 16 10:38:46.306416 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.306391 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qwmgq"] Jul 16 10:38:46.308140 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:38:46.308113 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c922ca_2a7f_4375_9f3a_28c033fecb1c.slice/crio-2799a784e7ae6ad33a9e362ecb8af461c48bb22d0851d54d87dba0e67b0f948f WatchSource:0}: Error finding container 2799a784e7ae6ad33a9e362ecb8af461c48bb22d0851d54d87dba0e67b0f948f: Status 404 returned error can't find the container with id 2799a784e7ae6ad33a9e362ecb8af461c48bb22d0851d54d87dba0e67b0f948f Jul 16 10:38:46.366223 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.366151 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" event={"ID":"99c922ca-2a7f-4375-9f3a-28c033fecb1c","Type":"ContainerStarted","Data":"2799a784e7ae6ad33a9e362ecb8af461c48bb22d0851d54d87dba0e67b0f948f"} Jul 16 10:38:46.400304 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.400274 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-prcsn"] Jul 16 10:38:46.402715 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.402682 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-prcsn" Jul 16 10:38:46.418184 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.418164 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-prcsn"] Jul 16 10:38:46.510914 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.510881 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d4r6\" (UniqueName: \"kubernetes.io/projected/6cb0bee4-8510-446a-a024-d09554bd1453-kube-api-access-2d4r6\") pod \"kuadrant-operator-catalog-prcsn\" (UID: \"6cb0bee4-8510-446a-a024-d09554bd1453\") " pod="kuadrant-system/kuadrant-operator-catalog-prcsn" Jul 16 10:38:46.611274 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.611238 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d4r6\" (UniqueName: \"kubernetes.io/projected/6cb0bee4-8510-446a-a024-d09554bd1453-kube-api-access-2d4r6\") pod \"kuadrant-operator-catalog-prcsn\" (UID: \"6cb0bee4-8510-446a-a024-d09554bd1453\") " pod="kuadrant-system/kuadrant-operator-catalog-prcsn" Jul 16 10:38:46.620979 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.620911 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d4r6\" (UniqueName: \"kubernetes.io/projected/6cb0bee4-8510-446a-a024-d09554bd1453-kube-api-access-2d4r6\") pod \"kuadrant-operator-catalog-prcsn\" (UID: \"6cb0bee4-8510-446a-a024-d09554bd1453\") " pod="kuadrant-system/kuadrant-operator-catalog-prcsn" Jul 16 10:38:46.711997 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.711956 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-prcsn" Jul 16 10:38:46.838812 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:46.838784 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-prcsn"] Jul 16 10:38:46.842069 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:38:46.842040 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb0bee4_8510_446a_a024_d09554bd1453.slice/crio-17c6d0b3dfd1802918fd6e85c93f540f23447a0c7287e4416af1d629d24e85ce WatchSource:0}: Error finding container 17c6d0b3dfd1802918fd6e85c93f540f23447a0c7287e4416af1d629d24e85ce: Status 404 returned error can't find the container with id 17c6d0b3dfd1802918fd6e85c93f540f23447a0c7287e4416af1d629d24e85ce Jul 16 10:38:47.372748 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:47.372708 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-prcsn" event={"ID":"6cb0bee4-8510-446a-a024-d09554bd1453","Type":"ContainerStarted","Data":"17c6d0b3dfd1802918fd6e85c93f540f23447a0c7287e4416af1d629d24e85ce"} Jul 16 10:38:49.384723 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:49.384677 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" event={"ID":"99c922ca-2a7f-4375-9f3a-28c033fecb1c","Type":"ContainerStarted","Data":"f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2"} Jul 16 10:38:49.385165 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:49.384824 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" podUID="99c922ca-2a7f-4375-9f3a-28c033fecb1c" containerName="registry-server" containerID="cri-o://f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2" gracePeriod=2 Jul 16 10:38:49.386285 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:49.386256 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-prcsn" event={"ID":"6cb0bee4-8510-446a-a024-d09554bd1453","Type":"ContainerStarted","Data":"f87883649a9f6b053ae6ae0afe78b530e7e0b4c27846f59d0dc7f3403a16fda1"} Jul 16 10:38:49.405633 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:49.405590 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" podStartSLOduration=2.253476884 podStartE2EDuration="4.405575958s" totalImagesPullingTime="2.152099074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:38:49.403837832 +0000 UTC m=+384.083102811" watchObservedRunningTime="2026-07-16 10:38:49.405575958 +0000 UTC m=+384.084840939" Jul 16 10:38:49.423355 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:49.423307 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-prcsn" podStartSLOduration=1.808437925 podStartE2EDuration="3.423291522s" totalImagesPullingTime="1.614853597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:38:49.422415998 +0000 UTC m=+384.101680989" watchObservedRunningTime="2026-07-16 10:38:49.423291522 +0000 UTC m=+384.102556501" Jul 16 10:38:49.624171 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:49.624144 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" Jul 16 10:38:49.738901 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:49.738826 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs96r\" (UniqueName: \"kubernetes.io/projected/99c922ca-2a7f-4375-9f3a-28c033fecb1c-kube-api-access-fs96r\") pod \"99c922ca-2a7f-4375-9f3a-28c033fecb1c\" (UID: \"99c922ca-2a7f-4375-9f3a-28c033fecb1c\") " Jul 16 10:38:49.740911 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:49.740886 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99c922ca-2a7f-4375-9f3a-28c033fecb1c-kube-api-access-fs96r" (OuterVolumeSpecName: "kube-api-access-fs96r") pod "99c922ca-2a7f-4375-9f3a-28c033fecb1c" (UID: "99c922ca-2a7f-4375-9f3a-28c033fecb1c"). InnerVolumeSpecName "kube-api-access-fs96r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:38:49.839492 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:49.839464 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs96r\" (UniqueName: \"kubernetes.io/projected/99c922ca-2a7f-4375-9f3a-28c033fecb1c-kube-api-access-fs96r\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:38:50.391569 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:50.391536 2581 generic.go:356] "Generic (PLEG): container finished" podID="99c922ca-2a7f-4375-9f3a-28c033fecb1c" containerID="f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2" exitCode=0 Jul 16 10:38:50.392024 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:50.391578 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" Jul 16 10:38:50.392024 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:50.391629 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" event={"ID":"99c922ca-2a7f-4375-9f3a-28c033fecb1c","Type":"ContainerDied","Data":"f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2"} Jul 16 10:38:50.392024 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:50.391682 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qwmgq" event={"ID":"99c922ca-2a7f-4375-9f3a-28c033fecb1c","Type":"ContainerDied","Data":"2799a784e7ae6ad33a9e362ecb8af461c48bb22d0851d54d87dba0e67b0f948f"} Jul 16 10:38:50.392024 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:50.391735 2581 scope.go:117] "RemoveContainer" containerID="f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2" Jul 16 10:38:50.400350 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:50.400330 2581 scope.go:117] "RemoveContainer" containerID="f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2" Jul 16 10:38:50.400601 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:38:50.400583 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2\": container with ID starting with f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2 not found: ID does not exist" containerID="f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2" Jul 16 10:38:50.400658 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:50.400611 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2"} err="failed to get container status \"f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2\": rpc error: code = NotFound desc = could not find container \"f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2\": container with ID starting with f1309b4b221643b93913845000c7f6a6d4967cac00cd1c2f5f4bed8f621337b2 not found: ID does not exist" Jul 16 10:38:50.410651 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:50.410631 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qwmgq"] Jul 16 10:38:50.412850 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:50.412831 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qwmgq"] Jul 16 10:38:51.804971 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:51.804942 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99c922ca-2a7f-4375-9f3a-28c033fecb1c" path="/var/lib/kubelet/pods/99c922ca-2a7f-4375-9f3a-28c033fecb1c/volumes" Jul 16 10:38:56.712633 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:56.712600 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-prcsn" Jul 16 10:38:56.713035 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:56.712657 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-prcsn" Jul 16 10:38:56.734446 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:56.734419 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-prcsn" Jul 16 10:38:57.444137 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:38:57.444109 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-prcsn" Jul 16 10:39:01.649761 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.649718 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr"] Jul 16 10:39:01.650204 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.650058 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99c922ca-2a7f-4375-9f3a-28c033fecb1c" containerName="registry-server" Jul 16 10:39:01.650204 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.650069 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="99c922ca-2a7f-4375-9f3a-28c033fecb1c" containerName="registry-server" Jul 16 10:39:01.650204 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.650129 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="99c922ca-2a7f-4375-9f3a-28c033fecb1c" containerName="registry-server" Jul 16 10:39:01.654421 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.654404 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:01.657808 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.657789 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gktxk\"" Jul 16 10:39:01.665465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.665443 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr"] Jul 16 10:39:01.728461 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.728425 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:01.728624 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.728471 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg8wn\" (UniqueName: \"kubernetes.io/projected/f80cc46e-073c-48cb-8cff-0b8fe5db5796-kube-api-access-vg8wn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:01.728624 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.728531 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:01.829569 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.829538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg8wn\" (UniqueName: \"kubernetes.io/projected/f80cc46e-073c-48cb-8cff-0b8fe5db5796-kube-api-access-vg8wn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:01.829784 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.829596 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:01.829784 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.829632 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:01.829971 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.829955 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:01.830089 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.830070 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:01.839579 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.839555 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg8wn\" (UniqueName: \"kubernetes.io/projected/f80cc46e-073c-48cb-8cff-0b8fe5db5796-kube-api-access-vg8wn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:01.962944 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:01.962849 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:02.080551 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.080522 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x"] Jul 16 10:39:02.084966 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.084949 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:02.101184 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.101160 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr"] Jul 16 10:39:02.104340 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.104320 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x"] Jul 16 10:39:02.106045 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:02.105984 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf80cc46e_073c_48cb_8cff_0b8fe5db5796.slice/crio-cf44bbd1b2a9ed274f77fe4da2ceaf0021e31b375145cae46da2fb750cc47176 WatchSource:0}: Error finding container cf44bbd1b2a9ed274f77fe4da2ceaf0021e31b375145cae46da2fb750cc47176: Status 404 returned error can't find the container with id cf44bbd1b2a9ed274f77fe4da2ceaf0021e31b375145cae46da2fb750cc47176 Jul 16 10:39:02.132082 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.132059 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:02.132194 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.132157 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:02.132194 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.132177 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5whcm\" (UniqueName: \"kubernetes.io/projected/dac76e78-2e12-48cf-a721-5b91709d6136-kube-api-access-5whcm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:02.232847 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.232783 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:02.232847 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.232817 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5whcm\" (UniqueName: \"kubernetes.io/projected/dac76e78-2e12-48cf-a721-5b91709d6136-kube-api-access-5whcm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:02.232847 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.232843 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:02.233153 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.233134 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:02.233193 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.233143 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:02.248181 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.248154 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5whcm\" (UniqueName: \"kubernetes.io/projected/dac76e78-2e12-48cf-a721-5b91709d6136-kube-api-access-5whcm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:02.394526 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.394499 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:02.450157 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.449940 2581 generic.go:356] "Generic (PLEG): container finished" podID="f80cc46e-073c-48cb-8cff-0b8fe5db5796" containerID="317d7668e599184676350392223d91a5b03a1afa24157dcd4af2a84bb784874a" exitCode=0 Jul 16 10:39:02.450157 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.450068 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" event={"ID":"f80cc46e-073c-48cb-8cff-0b8fe5db5796","Type":"ContainerDied","Data":"317d7668e599184676350392223d91a5b03a1afa24157dcd4af2a84bb784874a"} Jul 16 10:39:02.450157 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.450121 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" event={"ID":"f80cc46e-073c-48cb-8cff-0b8fe5db5796","Type":"ContainerStarted","Data":"cf44bbd1b2a9ed274f77fe4da2ceaf0021e31b375145cae46da2fb750cc47176"} Jul 16 10:39:02.475802 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.475771 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5"] Jul 16 10:39:02.482979 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.482921 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:02.496024 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.496003 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5"] Jul 16 10:39:02.531019 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.530993 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x"] Jul 16 10:39:02.533626 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:02.533602 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddac76e78_2e12_48cf_a721_5b91709d6136.slice/crio-ea13e8874fe527e7e6f374ac7ec1525729fda78f5d94423ec289ba641299c417 WatchSource:0}: Error finding container ea13e8874fe527e7e6f374ac7ec1525729fda78f5d94423ec289ba641299c417: Status 404 returned error can't find the container with id ea13e8874fe527e7e6f374ac7ec1525729fda78f5d94423ec289ba641299c417 Jul 16 10:39:02.534361 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.534326 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:02.534361 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.534357 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:02.534477 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.534416 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz4hr\" (UniqueName: \"kubernetes.io/projected/209bd80e-ba3c-406d-92f8-27f1b6d506e6-kube-api-access-mz4hr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:02.635508 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.635478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:02.635611 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.635514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:02.635650 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.635632 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz4hr\" (UniqueName: \"kubernetes.io/projected/209bd80e-ba3c-406d-92f8-27f1b6d506e6-kube-api-access-mz4hr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:02.635831 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.635815 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:02.635916 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.635901 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:02.644825 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.644803 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz4hr\" (UniqueName: \"kubernetes.io/projected/209bd80e-ba3c-406d-92f8-27f1b6d506e6-kube-api-access-mz4hr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:02.792225 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.792142 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:02.925496 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:02.925472 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5"] Jul 16 10:39:02.927040 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:02.927011 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209bd80e_ba3c_406d_92f8_27f1b6d506e6.slice/crio-395e6323a57275945242dbdad5e80fff25be513ed2aaeecd8985542609d6cf1e WatchSource:0}: Error finding container 395e6323a57275945242dbdad5e80fff25be513ed2aaeecd8985542609d6cf1e: Status 404 returned error can't find the container with id 395e6323a57275945242dbdad5e80fff25be513ed2aaeecd8985542609d6cf1e Jul 16 10:39:03.072339 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.072316 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7"] Jul 16 10:39:03.075555 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.075537 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:03.088306 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.088286 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7"] Jul 16 10:39:03.139226 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.139191 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:03.139384 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.139241 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:03.139384 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.139299 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lrm5\" (UniqueName: \"kubernetes.io/projected/460d2887-3a19-498b-a95d-e9839bf5c256-kube-api-access-9lrm5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:03.239961 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.239930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:03.240125 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.239990 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lrm5\" (UniqueName: \"kubernetes.io/projected/460d2887-3a19-498b-a95d-e9839bf5c256-kube-api-access-9lrm5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:03.240125 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.240040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:03.240305 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.240283 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:03.240368 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.240351 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:03.251119 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.251089 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lrm5\" (UniqueName: \"kubernetes.io/projected/460d2887-3a19-498b-a95d-e9839bf5c256-kube-api-access-9lrm5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:03.393011 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.392945 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:03.456909 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.456873 2581 generic.go:356] "Generic (PLEG): container finished" podID="dac76e78-2e12-48cf-a721-5b91709d6136" containerID="3732691728b674ffd5a66e6b681cda2b015c89f5cc100eff862ee8c298416417" exitCode=0 Jul 16 10:39:03.457061 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.456965 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" event={"ID":"dac76e78-2e12-48cf-a721-5b91709d6136","Type":"ContainerDied","Data":"3732691728b674ffd5a66e6b681cda2b015c89f5cc100eff862ee8c298416417"} Jul 16 10:39:03.457061 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.457052 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" event={"ID":"dac76e78-2e12-48cf-a721-5b91709d6136","Type":"ContainerStarted","Data":"ea13e8874fe527e7e6f374ac7ec1525729fda78f5d94423ec289ba641299c417"} Jul 16 10:39:03.458578 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.458558 2581 generic.go:356] "Generic (PLEG): container finished" podID="209bd80e-ba3c-406d-92f8-27f1b6d506e6" containerID="4b6fc5379a7399392327b160d174d69ecc93314c28b3971cdcb4adb6a1673264" exitCode=0 Jul 16 10:39:03.458745 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.458614 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" event={"ID":"209bd80e-ba3c-406d-92f8-27f1b6d506e6","Type":"ContainerDied","Data":"4b6fc5379a7399392327b160d174d69ecc93314c28b3971cdcb4adb6a1673264"} Jul 16 10:39:03.458745 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.458649 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" event={"ID":"209bd80e-ba3c-406d-92f8-27f1b6d506e6","Type":"ContainerStarted","Data":"395e6323a57275945242dbdad5e80fff25be513ed2aaeecd8985542609d6cf1e"} Jul 16 10:39:03.568955 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:03.568931 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7"] Jul 16 10:39:03.570727 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:03.570698 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod460d2887_3a19_498b_a95d_e9839bf5c256.slice/crio-e69033781f30b29dd69c2dd410cfdef9bcc75fa4382de8a17fda1874841e66a3 WatchSource:0}: Error finding container e69033781f30b29dd69c2dd410cfdef9bcc75fa4382de8a17fda1874841e66a3: Status 404 returned error can't find the container with id e69033781f30b29dd69c2dd410cfdef9bcc75fa4382de8a17fda1874841e66a3 Jul 16 10:39:04.464808 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:04.464773 2581 generic.go:356] "Generic (PLEG): container finished" podID="f80cc46e-073c-48cb-8cff-0b8fe5db5796" containerID="094a82c61a15ebf09a4b16f36c03614cb513c5008cca6627a41e11f6c5395d6f" exitCode=0 Jul 16 10:39:04.465209 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:04.464858 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" event={"ID":"f80cc46e-073c-48cb-8cff-0b8fe5db5796","Type":"ContainerDied","Data":"094a82c61a15ebf09a4b16f36c03614cb513c5008cca6627a41e11f6c5395d6f"} Jul 16 10:39:04.466643 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:04.466608 2581 generic.go:356] "Generic (PLEG): container finished" podID="dac76e78-2e12-48cf-a721-5b91709d6136" containerID="9ccaa48ab2ff0c00e715c23ccfebc8a2229d646b27a32b3503f38d75a4f39594" exitCode=0 Jul 16 10:39:04.466731 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:04.466700 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" event={"ID":"dac76e78-2e12-48cf-a721-5b91709d6136","Type":"ContainerDied","Data":"9ccaa48ab2ff0c00e715c23ccfebc8a2229d646b27a32b3503f38d75a4f39594"} Jul 16 10:39:04.468195 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:04.468171 2581 generic.go:356] "Generic (PLEG): container finished" podID="460d2887-3a19-498b-a95d-e9839bf5c256" containerID="e5d90d053efabf12c54ba4b3127594abe151c671d3731b548c79116bb0982b2a" exitCode=0 Jul 16 10:39:04.468290 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:04.468263 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" event={"ID":"460d2887-3a19-498b-a95d-e9839bf5c256","Type":"ContainerDied","Data":"e5d90d053efabf12c54ba4b3127594abe151c671d3731b548c79116bb0982b2a"} Jul 16 10:39:04.468401 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:04.468317 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" event={"ID":"460d2887-3a19-498b-a95d-e9839bf5c256","Type":"ContainerStarted","Data":"e69033781f30b29dd69c2dd410cfdef9bcc75fa4382de8a17fda1874841e66a3"} Jul 16 10:39:04.470217 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:04.470199 2581 generic.go:356] "Generic (PLEG): container finished" podID="209bd80e-ba3c-406d-92f8-27f1b6d506e6" containerID="ac4db0f4e9c237b9e38561b03e9be85d5baeb8c02e7674ad7c0660f99c35c53c" exitCode=0 Jul 16 10:39:04.470287 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:04.470262 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" event={"ID":"209bd80e-ba3c-406d-92f8-27f1b6d506e6","Type":"ContainerDied","Data":"ac4db0f4e9c237b9e38561b03e9be85d5baeb8c02e7674ad7c0660f99c35c53c"} Jul 16 10:39:05.476763 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:05.476724 2581 generic.go:356] "Generic (PLEG): container finished" podID="f80cc46e-073c-48cb-8cff-0b8fe5db5796" containerID="9e5409456966f60e68a9d26acd397f3978160243a8f7649ddcc8b81bd4ca1f43" exitCode=0 Jul 16 10:39:05.477200 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:05.476801 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" event={"ID":"f80cc46e-073c-48cb-8cff-0b8fe5db5796","Type":"ContainerDied","Data":"9e5409456966f60e68a9d26acd397f3978160243a8f7649ddcc8b81bd4ca1f43"} Jul 16 10:39:05.478861 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:05.478836 2581 generic.go:356] "Generic (PLEG): container finished" podID="dac76e78-2e12-48cf-a721-5b91709d6136" containerID="cfcf4715edd4f54d87f65476a97654c15af317bd0d92aab808856f265ebe1e2e" exitCode=0 Jul 16 10:39:05.478985 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:05.478916 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" event={"ID":"dac76e78-2e12-48cf-a721-5b91709d6136","Type":"ContainerDied","Data":"cfcf4715edd4f54d87f65476a97654c15af317bd0d92aab808856f265ebe1e2e"} Jul 16 10:39:05.480468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:05.480447 2581 generic.go:356] "Generic (PLEG): container finished" podID="460d2887-3a19-498b-a95d-e9839bf5c256" containerID="15befb4f964bdd5291f8c6f399ddb1299eefeebc993f8f01d299c60588b68814" exitCode=0 Jul 16 10:39:05.480567 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:05.480528 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" event={"ID":"460d2887-3a19-498b-a95d-e9839bf5c256","Type":"ContainerDied","Data":"15befb4f964bdd5291f8c6f399ddb1299eefeebc993f8f01d299c60588b68814"} Jul 16 10:39:05.482771 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:05.482752 2581 generic.go:356] "Generic (PLEG): container finished" podID="209bd80e-ba3c-406d-92f8-27f1b6d506e6" containerID="70d0773392e183fa11e46ba9902d9a277bb9645998a2549fa692040970c8fa90" exitCode=0 Jul 16 10:39:05.482874 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:05.482790 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" event={"ID":"209bd80e-ba3c-406d-92f8-27f1b6d506e6","Type":"ContainerDied","Data":"70d0773392e183fa11e46ba9902d9a277bb9645998a2549fa692040970c8fa90"} Jul 16 10:39:06.489306 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.489273 2581 generic.go:356] "Generic (PLEG): container finished" podID="460d2887-3a19-498b-a95d-e9839bf5c256" containerID="6bff505c68f4bb2be68b88a804b92008d9555d9b8550a56d2ab16c69d95e7e17" exitCode=0 Jul 16 10:39:06.489798 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.489359 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" event={"ID":"460d2887-3a19-498b-a95d-e9839bf5c256","Type":"ContainerDied","Data":"6bff505c68f4bb2be68b88a804b92008d9555d9b8550a56d2ab16c69d95e7e17"} Jul 16 10:39:06.618658 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.618623 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:06.665769 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.665735 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-bundle\") pod \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " Jul 16 10:39:06.665917 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.665786 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mz4hr\" (UniqueName: \"kubernetes.io/projected/209bd80e-ba3c-406d-92f8-27f1b6d506e6-kube-api-access-mz4hr\") pod \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " Jul 16 10:39:06.665917 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.665845 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-util\") pod \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\" (UID: \"209bd80e-ba3c-406d-92f8-27f1b6d506e6\") " Jul 16 10:39:06.666521 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.666494 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-bundle" (OuterVolumeSpecName: "bundle") pod "209bd80e-ba3c-406d-92f8-27f1b6d506e6" (UID: "209bd80e-ba3c-406d-92f8-27f1b6d506e6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:39:06.668553 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.668527 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/209bd80e-ba3c-406d-92f8-27f1b6d506e6-kube-api-access-mz4hr" (OuterVolumeSpecName: "kube-api-access-mz4hr") pod "209bd80e-ba3c-406d-92f8-27f1b6d506e6" (UID: "209bd80e-ba3c-406d-92f8-27f1b6d506e6"). InnerVolumeSpecName "kube-api-access-mz4hr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:39:06.672277 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.672255 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-util" (OuterVolumeSpecName: "util") pod "209bd80e-ba3c-406d-92f8-27f1b6d506e6" (UID: "209bd80e-ba3c-406d-92f8-27f1b6d506e6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:39:06.681101 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.681079 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:06.683775 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.683760 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:06.766829 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.766760 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-util\") pod \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " Jul 16 10:39:06.766829 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.766798 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vg8wn\" (UniqueName: \"kubernetes.io/projected/f80cc46e-073c-48cb-8cff-0b8fe5db5796-kube-api-access-vg8wn\") pod \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " Jul 16 10:39:06.767043 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.766840 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-bundle\") pod \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\" (UID: \"f80cc46e-073c-48cb-8cff-0b8fe5db5796\") " Jul 16 10:39:06.767043 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.766858 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-bundle\") pod \"dac76e78-2e12-48cf-a721-5b91709d6136\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " Jul 16 10:39:06.767043 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.766876 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-util\") pod \"dac76e78-2e12-48cf-a721-5b91709d6136\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " Jul 16 10:39:06.767043 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.766930 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5whcm\" (UniqueName: \"kubernetes.io/projected/dac76e78-2e12-48cf-a721-5b91709d6136-kube-api-access-5whcm\") pod \"dac76e78-2e12-48cf-a721-5b91709d6136\" (UID: \"dac76e78-2e12-48cf-a721-5b91709d6136\") " Jul 16 10:39:06.767227 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.767151 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-util\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:06.767227 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.767167 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/209bd80e-ba3c-406d-92f8-27f1b6d506e6-bundle\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:06.767227 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.767180 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz4hr\" (UniqueName: \"kubernetes.io/projected/209bd80e-ba3c-406d-92f8-27f1b6d506e6-kube-api-access-mz4hr\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:06.767374 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.767352 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-bundle" (OuterVolumeSpecName: "bundle") pod "f80cc46e-073c-48cb-8cff-0b8fe5db5796" (UID: "f80cc46e-073c-48cb-8cff-0b8fe5db5796"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:39:06.767486 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.767460 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-bundle" (OuterVolumeSpecName: "bundle") pod "dac76e78-2e12-48cf-a721-5b91709d6136" (UID: "dac76e78-2e12-48cf-a721-5b91709d6136"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:39:06.768917 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.768895 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f80cc46e-073c-48cb-8cff-0b8fe5db5796-kube-api-access-vg8wn" (OuterVolumeSpecName: "kube-api-access-vg8wn") pod "f80cc46e-073c-48cb-8cff-0b8fe5db5796" (UID: "f80cc46e-073c-48cb-8cff-0b8fe5db5796"). InnerVolumeSpecName "kube-api-access-vg8wn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:39:06.769023 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.768963 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dac76e78-2e12-48cf-a721-5b91709d6136-kube-api-access-5whcm" (OuterVolumeSpecName: "kube-api-access-5whcm") pod "dac76e78-2e12-48cf-a721-5b91709d6136" (UID: "dac76e78-2e12-48cf-a721-5b91709d6136"). InnerVolumeSpecName "kube-api-access-5whcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:39:06.772730 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.772703 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-util" (OuterVolumeSpecName: "util") pod "dac76e78-2e12-48cf-a721-5b91709d6136" (UID: "dac76e78-2e12-48cf-a721-5b91709d6136"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:39:06.772997 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.772976 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-util" (OuterVolumeSpecName: "util") pod "f80cc46e-073c-48cb-8cff-0b8fe5db5796" (UID: "f80cc46e-073c-48cb-8cff-0b8fe5db5796"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:39:06.869053 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.869023 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5whcm\" (UniqueName: \"kubernetes.io/projected/dac76e78-2e12-48cf-a721-5b91709d6136-kube-api-access-5whcm\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:06.869183 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.869067 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-util\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:06.869183 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.869091 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg8wn\" (UniqueName: \"kubernetes.io/projected/f80cc46e-073c-48cb-8cff-0b8fe5db5796-kube-api-access-vg8wn\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:06.869183 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.869107 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f80cc46e-073c-48cb-8cff-0b8fe5db5796-bundle\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:06.869183 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.869118 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-bundle\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:06.869183 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:06.869126 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dac76e78-2e12-48cf-a721-5b91709d6136-util\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:07.496236 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.496208 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" Jul 16 10:39:07.496635 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.496236 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5" event={"ID":"209bd80e-ba3c-406d-92f8-27f1b6d506e6","Type":"ContainerDied","Data":"395e6323a57275945242dbdad5e80fff25be513ed2aaeecd8985542609d6cf1e"} Jul 16 10:39:07.496635 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.496281 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="395e6323a57275945242dbdad5e80fff25be513ed2aaeecd8985542609d6cf1e" Jul 16 10:39:07.498155 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.498124 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" event={"ID":"f80cc46e-073c-48cb-8cff-0b8fe5db5796","Type":"ContainerDied","Data":"cf44bbd1b2a9ed274f77fe4da2ceaf0021e31b375145cae46da2fb750cc47176"} Jul 16 10:39:07.498269 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.498163 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf44bbd1b2a9ed274f77fe4da2ceaf0021e31b375145cae46da2fb750cc47176" Jul 16 10:39:07.498269 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.498133 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr" Jul 16 10:39:07.500046 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.500027 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" Jul 16 10:39:07.500153 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.500054 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x" event={"ID":"dac76e78-2e12-48cf-a721-5b91709d6136","Type":"ContainerDied","Data":"ea13e8874fe527e7e6f374ac7ec1525729fda78f5d94423ec289ba641299c417"} Jul 16 10:39:07.500153 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.500091 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea13e8874fe527e7e6f374ac7ec1525729fda78f5d94423ec289ba641299c417" Jul 16 10:39:07.626398 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.626377 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:07.674133 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.674110 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-util\") pod \"460d2887-3a19-498b-a95d-e9839bf5c256\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " Jul 16 10:39:07.674228 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.674164 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-bundle\") pod \"460d2887-3a19-498b-a95d-e9839bf5c256\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " Jul 16 10:39:07.674228 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.674198 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lrm5\" (UniqueName: \"kubernetes.io/projected/460d2887-3a19-498b-a95d-e9839bf5c256-kube-api-access-9lrm5\") pod \"460d2887-3a19-498b-a95d-e9839bf5c256\" (UID: \"460d2887-3a19-498b-a95d-e9839bf5c256\") " Jul 16 10:39:07.674649 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.674623 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-bundle" (OuterVolumeSpecName: "bundle") pod "460d2887-3a19-498b-a95d-e9839bf5c256" (UID: "460d2887-3a19-498b-a95d-e9839bf5c256"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:39:07.676208 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.676184 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/460d2887-3a19-498b-a95d-e9839bf5c256-kube-api-access-9lrm5" (OuterVolumeSpecName: "kube-api-access-9lrm5") pod "460d2887-3a19-498b-a95d-e9839bf5c256" (UID: "460d2887-3a19-498b-a95d-e9839bf5c256"). InnerVolumeSpecName "kube-api-access-9lrm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:39:07.680304 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.680246 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-util" (OuterVolumeSpecName: "util") pod "460d2887-3a19-498b-a95d-e9839bf5c256" (UID: "460d2887-3a19-498b-a95d-e9839bf5c256"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:39:07.775218 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.775145 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-util\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:07.775218 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.775170 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/460d2887-3a19-498b-a95d-e9839bf5c256-bundle\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:07.775218 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:07.775182 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lrm5\" (UniqueName: \"kubernetes.io/projected/460d2887-3a19-498b-a95d-e9839bf5c256-kube-api-access-9lrm5\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:08.507112 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:08.507080 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" event={"ID":"460d2887-3a19-498b-a95d-e9839bf5c256","Type":"ContainerDied","Data":"e69033781f30b29dd69c2dd410cfdef9bcc75fa4382de8a17fda1874841e66a3"} Jul 16 10:39:08.507488 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:08.507120 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7" Jul 16 10:39:08.507488 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:08.507124 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e69033781f30b29dd69c2dd410cfdef9bcc75fa4382de8a17fda1874841e66a3" Jul 16 10:39:18.513862 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.513825 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx"] Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514123 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="460d2887-3a19-498b-a95d-e9839bf5c256" containerName="extract" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514134 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="460d2887-3a19-498b-a95d-e9839bf5c256" containerName="extract" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514146 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dac76e78-2e12-48cf-a721-5b91709d6136" containerName="pull" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514151 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dac76e78-2e12-48cf-a721-5b91709d6136" containerName="pull" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514162 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="460d2887-3a19-498b-a95d-e9839bf5c256" containerName="util" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514168 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="460d2887-3a19-498b-a95d-e9839bf5c256" containerName="util" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514174 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dac76e78-2e12-48cf-a721-5b91709d6136" containerName="extract" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514179 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dac76e78-2e12-48cf-a721-5b91709d6136" containerName="extract" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514186 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dac76e78-2e12-48cf-a721-5b91709d6136" containerName="util" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514190 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dac76e78-2e12-48cf-a721-5b91709d6136" containerName="util" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514197 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f80cc46e-073c-48cb-8cff-0b8fe5db5796" containerName="pull" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514202 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f80cc46e-073c-48cb-8cff-0b8fe5db5796" containerName="pull" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514208 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f80cc46e-073c-48cb-8cff-0b8fe5db5796" containerName="extract" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514212 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f80cc46e-073c-48cb-8cff-0b8fe5db5796" containerName="extract" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514222 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="209bd80e-ba3c-406d-92f8-27f1b6d506e6" containerName="pull" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514227 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="209bd80e-ba3c-406d-92f8-27f1b6d506e6" containerName="pull" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514232 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="209bd80e-ba3c-406d-92f8-27f1b6d506e6" containerName="extract" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514236 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="209bd80e-ba3c-406d-92f8-27f1b6d506e6" containerName="extract" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514242 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f80cc46e-073c-48cb-8cff-0b8fe5db5796" containerName="util" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514247 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f80cc46e-073c-48cb-8cff-0b8fe5db5796" containerName="util" Jul 16 10:39:18.514242 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514253 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="460d2887-3a19-498b-a95d-e9839bf5c256" containerName="pull" Jul 16 10:39:18.514860 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514258 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="460d2887-3a19-498b-a95d-e9839bf5c256" containerName="pull" Jul 16 10:39:18.514860 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514265 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="209bd80e-ba3c-406d-92f8-27f1b6d506e6" containerName="util" Jul 16 10:39:18.514860 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514270 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="209bd80e-ba3c-406d-92f8-27f1b6d506e6" containerName="util" Jul 16 10:39:18.514860 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514313 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="209bd80e-ba3c-406d-92f8-27f1b6d506e6" containerName="extract" Jul 16 10:39:18.514860 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514321 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f80cc46e-073c-48cb-8cff-0b8fe5db5796" containerName="extract" Jul 16 10:39:18.514860 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514326 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dac76e78-2e12-48cf-a721-5b91709d6136" containerName="extract" Jul 16 10:39:18.514860 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.514334 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="460d2887-3a19-498b-a95d-e9839bf5c256" containerName="extract" Jul 16 10:39:18.516891 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.516875 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:18.522323 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.522300 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-fp8c9\"" Jul 16 10:39:18.558282 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.558246 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5brw\" (UniqueName: \"kubernetes.io/projected/73f7b472-2485-4225-b915-d4414e2816df-kube-api-access-g5brw\") pod \"limitador-operator-controller-manager-859d7b78bd-rpzpx\" (UID: \"73f7b472-2485-4225-b915-d4414e2816df\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:18.560993 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.560969 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx"] Jul 16 10:39:18.659415 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.659374 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5brw\" (UniqueName: \"kubernetes.io/projected/73f7b472-2485-4225-b915-d4414e2816df-kube-api-access-g5brw\") pod \"limitador-operator-controller-manager-859d7b78bd-rpzpx\" (UID: \"73f7b472-2485-4225-b915-d4414e2816df\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:18.702757 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.702721 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5brw\" (UniqueName: \"kubernetes.io/projected/73f7b472-2485-4225-b915-d4414e2816df-kube-api-access-g5brw\") pod \"limitador-operator-controller-manager-859d7b78bd-rpzpx\" (UID: \"73f7b472-2485-4225-b915-d4414e2816df\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:18.826680 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.826645 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:18.969097 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:18.969077 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx"] Jul 16 10:39:18.971233 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:18.971208 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73f7b472_2485_4225_b915_d4414e2816df.slice/crio-3948d519b72cf862a94a0eeac1f330b38d687fdf87fe65ede05cab041a32f257 WatchSource:0}: Error finding container 3948d519b72cf862a94a0eeac1f330b38d687fdf87fe65ede05cab041a32f257: Status 404 returned error can't find the container with id 3948d519b72cf862a94a0eeac1f330b38d687fdf87fe65ede05cab041a32f257 Jul 16 10:39:19.561814 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:19.561783 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" event={"ID":"73f7b472-2485-4225-b915-d4414e2816df","Type":"ContainerStarted","Data":"3948d519b72cf862a94a0eeac1f330b38d687fdf87fe65ede05cab041a32f257"} Jul 16 10:39:20.712494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.712463 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7487448f6b-2jtcs"] Jul 16 10:39:20.715662 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.715646 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.718871 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.718851 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qp7fd\"" Jul 16 10:39:20.720273 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.720259 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 10:39:20.720479 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.720457 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 10:39:20.720554 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.720498 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 10:39:20.720742 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.720725 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 10:39:20.721660 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.721640 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 10:39:20.721827 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.721808 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 10:39:20.727022 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.727007 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 10:39:20.728200 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.728186 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 10:39:20.741593 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.741572 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7487448f6b-2jtcs"] Jul 16 10:39:20.776936 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.776915 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54b45578-1fbb-4531-98f3-665a393abbdc-console-serving-cert\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.777033 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.776950 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-oauth-serving-cert\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.777033 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.777014 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8xwk\" (UniqueName: \"kubernetes.io/projected/54b45578-1fbb-4531-98f3-665a393abbdc-kube-api-access-f8xwk\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.777099 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.777055 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-console-config\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.777099 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.777088 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54b45578-1fbb-4531-98f3-665a393abbdc-console-oauth-config\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.777156 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.777103 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-service-ca\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.777156 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.777134 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-trusted-ca-bundle\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.878070 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.878037 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54b45578-1fbb-4531-98f3-665a393abbdc-console-serving-cert\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.878214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.878077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-oauth-serving-cert\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.878214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.878106 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8xwk\" (UniqueName: \"kubernetes.io/projected/54b45578-1fbb-4531-98f3-665a393abbdc-kube-api-access-f8xwk\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.878314 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.878230 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-console-config\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.878314 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.878273 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54b45578-1fbb-4531-98f3-665a393abbdc-console-oauth-config\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.878314 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.878290 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-service-ca\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.878314 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.878304 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-trusted-ca-bundle\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.878829 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.878807 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-oauth-serving-cert\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.878991 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.878969 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-console-config\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.879051 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.879041 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-service-ca\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.879156 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.879135 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54b45578-1fbb-4531-98f3-665a393abbdc-trusted-ca-bundle\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.880663 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.880643 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54b45578-1fbb-4531-98f3-665a393abbdc-console-oauth-config\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.880759 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.880738 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54b45578-1fbb-4531-98f3-665a393abbdc-console-serving-cert\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:20.888613 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:20.888592 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8xwk\" (UniqueName: \"kubernetes.io/projected/54b45578-1fbb-4531-98f3-665a393abbdc-kube-api-access-f8xwk\") pod \"console-7487448f6b-2jtcs\" (UID: \"54b45578-1fbb-4531-98f3-665a393abbdc\") " pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:21.024972 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.024939 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:21.161616 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.161588 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7487448f6b-2jtcs"] Jul 16 10:39:21.162920 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:21.162891 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b45578_1fbb_4531_98f3_665a393abbdc.slice/crio-f70c93fc3ac7c64cc4a9ad35b1d18b1441ef07a29b7284c56498c46888216464 WatchSource:0}: Error finding container f70c93fc3ac7c64cc4a9ad35b1d18b1441ef07a29b7284c56498c46888216464: Status 404 returned error can't find the container with id f70c93fc3ac7c64cc4a9ad35b1d18b1441ef07a29b7284c56498c46888216464 Jul 16 10:39:21.572408 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.572368 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" event={"ID":"73f7b472-2485-4225-b915-d4414e2816df","Type":"ContainerStarted","Data":"fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad"} Jul 16 10:39:21.572566 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.572520 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:21.573885 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.573862 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7487448f6b-2jtcs" event={"ID":"54b45578-1fbb-4531-98f3-665a393abbdc","Type":"ContainerStarted","Data":"3e25b97730514666d4f29f6a48a2252cf704bdf3af6208071d46047d13e56456"} Jul 16 10:39:21.573970 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.573900 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7487448f6b-2jtcs" event={"ID":"54b45578-1fbb-4531-98f3-665a393abbdc","Type":"ContainerStarted","Data":"f70c93fc3ac7c64cc4a9ad35b1d18b1441ef07a29b7284c56498c46888216464"} Jul 16 10:39:21.599098 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.599060 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" podStartSLOduration=2.039642993 podStartE2EDuration="3.599045336s" totalImagesPullingTime="1.559402343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:39:21.597642867 +0000 UTC m=+416.276907849" watchObservedRunningTime="2026-07-16 10:39:21.599045336 +0000 UTC m=+416.278310316" Jul 16 10:39:21.617169 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.617133 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7487448f6b-2jtcs" podStartSLOduration=1.617119415 podStartE2EDuration="1.617119415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:39:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:39:21.615427264 +0000 UTC m=+416.294692271" watchObservedRunningTime="2026-07-16 10:39:21.617119415 +0000 UTC m=+416.296384394" Jul 16 10:39:21.741924 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.741897 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984"] Jul 16 10:39:21.745255 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.745240 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984" Jul 16 10:39:21.747740 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.747715 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hh89h\"" Jul 16 10:39:21.747835 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.747822 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 16 10:39:21.758484 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.758464 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984"] Jul 16 10:39:21.786530 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.786506 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkxvv\" (UniqueName: \"kubernetes.io/projected/081940f1-f71d-4157-92f8-9c6eadd3e93d-kube-api-access-gkxvv\") pod \"dns-operator-controller-manager-864dd78c4c-95984\" (UID: \"081940f1-f71d-4157-92f8-9c6eadd3e93d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984" Jul 16 10:39:21.888052 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.887968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkxvv\" (UniqueName: \"kubernetes.io/projected/081940f1-f71d-4157-92f8-9c6eadd3e93d-kube-api-access-gkxvv\") pod \"dns-operator-controller-manager-864dd78c4c-95984\" (UID: \"081940f1-f71d-4157-92f8-9c6eadd3e93d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984" Jul 16 10:39:21.903524 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:21.903498 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkxvv\" (UniqueName: \"kubernetes.io/projected/081940f1-f71d-4157-92f8-9c6eadd3e93d-kube-api-access-gkxvv\") pod \"dns-operator-controller-manager-864dd78c4c-95984\" (UID: \"081940f1-f71d-4157-92f8-9c6eadd3e93d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984" Jul 16 10:39:22.055619 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:22.055587 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984" Jul 16 10:39:22.188494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:22.188466 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984"] Jul 16 10:39:22.190162 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:22.190135 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081940f1_f71d_4157_92f8_9c6eadd3e93d.slice/crio-91a65100a70f4693afaa5d13efd766db90bd0b18a5971f77d5bf4de81b072cf1 WatchSource:0}: Error finding container 91a65100a70f4693afaa5d13efd766db90bd0b18a5971f77d5bf4de81b072cf1: Status 404 returned error can't find the container with id 91a65100a70f4693afaa5d13efd766db90bd0b18a5971f77d5bf4de81b072cf1 Jul 16 10:39:22.580273 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:22.580230 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984" event={"ID":"081940f1-f71d-4157-92f8-9c6eadd3e93d","Type":"ContainerStarted","Data":"91a65100a70f4693afaa5d13efd766db90bd0b18a5971f77d5bf4de81b072cf1"} Jul 16 10:39:24.592560 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:24.592529 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984" event={"ID":"081940f1-f71d-4157-92f8-9c6eadd3e93d","Type":"ContainerStarted","Data":"aa7f93e3c7df1618d5a7ed5fd5b1c3fe20f20b9f1e03852e1d6eb558d44bc0ad"} Jul 16 10:39:24.592935 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:24.592641 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984" Jul 16 10:39:24.627289 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:24.627237 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984" podStartSLOduration=1.6597195839999999 podStartE2EDuration="3.627223111s" totalImagesPullingTime="1.967503527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:39:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:39:24.621183998 +0000 UTC m=+419.300448978" watchObservedRunningTime="2026-07-16 10:39:24.627223111 +0000 UTC m=+419.306488090" Jul 16 10:39:27.024179 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.024085 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv"] Jul 16 10:39:27.029066 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.029042 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:27.032027 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.032005 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-g4tqf\"" Jul 16 10:39:27.052260 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.052233 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv"] Jul 16 10:39:27.135392 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.135359 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0cc54cc-14db-4a51-a565-863cd7fae54d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" (UID: \"b0cc54cc-14db-4a51-a565-863cd7fae54d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:27.135392 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.135395 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km4ps\" (UniqueName: \"kubernetes.io/projected/b0cc54cc-14db-4a51-a565-863cd7fae54d-kube-api-access-km4ps\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" (UID: \"b0cc54cc-14db-4a51-a565-863cd7fae54d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:27.236442 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.236412 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0cc54cc-14db-4a51-a565-863cd7fae54d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" (UID: \"b0cc54cc-14db-4a51-a565-863cd7fae54d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:27.236581 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.236448 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km4ps\" (UniqueName: \"kubernetes.io/projected/b0cc54cc-14db-4a51-a565-863cd7fae54d-kube-api-access-km4ps\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" (UID: \"b0cc54cc-14db-4a51-a565-863cd7fae54d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:27.236800 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.236780 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0cc54cc-14db-4a51-a565-863cd7fae54d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" (UID: \"b0cc54cc-14db-4a51-a565-863cd7fae54d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:27.261164 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.261130 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km4ps\" (UniqueName: \"kubernetes.io/projected/b0cc54cc-14db-4a51-a565-863cd7fae54d-kube-api-access-km4ps\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" (UID: \"b0cc54cc-14db-4a51-a565-863cd7fae54d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:27.338159 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.338132 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:27.496503 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.496469 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv"] Jul 16 10:39:27.499199 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:27.499174 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0cc54cc_14db_4a51_a565_863cd7fae54d.slice/crio-aaf904415053c6a7131368fd71a0466699e6b313f7221a7b457649f9675c373a WatchSource:0}: Error finding container aaf904415053c6a7131368fd71a0466699e6b313f7221a7b457649f9675c373a: Status 404 returned error can't find the container with id aaf904415053c6a7131368fd71a0466699e6b313f7221a7b457649f9675c373a Jul 16 10:39:27.608582 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:27.608503 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" event={"ID":"b0cc54cc-14db-4a51-a565-863cd7fae54d","Type":"ContainerStarted","Data":"aaf904415053c6a7131368fd71a0466699e6b313f7221a7b457649f9675c373a"} Jul 16 10:39:30.747947 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:30.747912 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq"] Jul 16 10:39:30.751361 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:30.751344 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq" Jul 16 10:39:30.754585 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:30.754562 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-c2rd7\"" Jul 16 10:39:30.773538 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:30.773515 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq"] Jul 16 10:39:30.868659 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:30.868628 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz4d4\" (UniqueName: \"kubernetes.io/projected/c19d9c7e-db07-4b42-ac19-2932a6716ab3-kube-api-access-dz4d4\") pod \"authorino-operator-75cc8d5cb7-s6ndq\" (UID: \"c19d9c7e-db07-4b42-ac19-2932a6716ab3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq" Jul 16 10:39:30.969232 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:30.969196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz4d4\" (UniqueName: \"kubernetes.io/projected/c19d9c7e-db07-4b42-ac19-2932a6716ab3-kube-api-access-dz4d4\") pod \"authorino-operator-75cc8d5cb7-s6ndq\" (UID: \"c19d9c7e-db07-4b42-ac19-2932a6716ab3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq" Jul 16 10:39:30.984976 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:30.984945 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz4d4\" (UniqueName: \"kubernetes.io/projected/c19d9c7e-db07-4b42-ac19-2932a6716ab3-kube-api-access-dz4d4\") pod \"authorino-operator-75cc8d5cb7-s6ndq\" (UID: \"c19d9c7e-db07-4b42-ac19-2932a6716ab3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq" Jul 16 10:39:31.025897 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:31.025817 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:31.026045 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:31.025900 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:31.031474 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:31.031451 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:31.061677 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:31.061637 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq" Jul 16 10:39:31.211742 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:31.211702 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq"] Jul 16 10:39:31.220546 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:31.220518 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc19d9c7e_db07_4b42_ac19_2932a6716ab3.slice/crio-d6ac9f9f2ec30d9125f9f50fcaaedafca0c668462adacd69d8caecf6a4a8e90d WatchSource:0}: Error finding container d6ac9f9f2ec30d9125f9f50fcaaedafca0c668462adacd69d8caecf6a4a8e90d: Status 404 returned error can't find the container with id d6ac9f9f2ec30d9125f9f50fcaaedafca0c668462adacd69d8caecf6a4a8e90d Jul 16 10:39:31.630586 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:31.630552 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq" event={"ID":"c19d9c7e-db07-4b42-ac19-2932a6716ab3","Type":"ContainerStarted","Data":"d6ac9f9f2ec30d9125f9f50fcaaedafca0c668462adacd69d8caecf6a4a8e90d"} Jul 16 10:39:31.634407 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:31.634386 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7487448f6b-2jtcs" Jul 16 10:39:32.583626 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:32.583568 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:33.645099 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:33.645052 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" event={"ID":"b0cc54cc-14db-4a51-a565-863cd7fae54d","Type":"ContainerStarted","Data":"74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347"} Jul 16 10:39:33.645535 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:33.645323 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:33.681149 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:33.681104 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" podStartSLOduration=1.173877016 podStartE2EDuration="6.681089873s" totalImagesPullingTime="5.507212857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:39:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:39:33.678976442 +0000 UTC m=+428.358241421" watchObservedRunningTime="2026-07-16 10:39:33.681089873 +0000 UTC m=+428.360354851" Jul 16 10:39:35.600103 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:35.600070 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95984" Jul 16 10:39:35.656744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:35.656711 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq" event={"ID":"c19d9c7e-db07-4b42-ac19-2932a6716ab3","Type":"ContainerStarted","Data":"187fa48cdc7a505480340b72360c9092c75d1b804d96219c88ec55882c48c980"} Jul 16 10:39:35.656924 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:35.656830 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq" Jul 16 10:39:35.695497 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:35.695441 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq" podStartSLOduration=2.239020467 podStartE2EDuration="5.695424795s" totalImagesPullingTime="3.456404328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:39:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:39:35.692680559 +0000 UTC m=+430.371945540" watchObservedRunningTime="2026-07-16 10:39:35.695424795 +0000 UTC m=+430.374689773" Jul 16 10:39:36.192298 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.192266 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt"] Jul 16 10:39:36.197606 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.197586 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:36.233474 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.233415 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gktxk\"" Jul 16 10:39:36.233474 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.233453 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 16 10:39:36.234725 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.234700 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt"] Jul 16 10:39:36.238391 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.238376 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 16 10:39:36.319949 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.319919 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b97797a-f035-4886-a2c3-579fc6482430-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-ldstt\" (UID: \"5b97797a-f035-4886-a2c3-579fc6482430\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:36.320098 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.319969 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5b97797a-f035-4886-a2c3-579fc6482430-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-ldstt\" (UID: \"5b97797a-f035-4886-a2c3-579fc6482430\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:36.320098 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.319991 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9vdk\" (UniqueName: \"kubernetes.io/projected/5b97797a-f035-4886-a2c3-579fc6482430-kube-api-access-n9vdk\") pod \"kuadrant-console-plugin-66b5774985-ldstt\" (UID: \"5b97797a-f035-4886-a2c3-579fc6482430\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:36.420921 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.420886 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b97797a-f035-4886-a2c3-579fc6482430-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-ldstt\" (UID: \"5b97797a-f035-4886-a2c3-579fc6482430\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:36.421084 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.420948 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5b97797a-f035-4886-a2c3-579fc6482430-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-ldstt\" (UID: \"5b97797a-f035-4886-a2c3-579fc6482430\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:36.421084 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.421019 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9vdk\" (UniqueName: \"kubernetes.io/projected/5b97797a-f035-4886-a2c3-579fc6482430-kube-api-access-n9vdk\") pod \"kuadrant-console-plugin-66b5774985-ldstt\" (UID: \"5b97797a-f035-4886-a2c3-579fc6482430\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:36.421159 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:36.421083 2581 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 16 10:39:36.421159 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:36.421148 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b97797a-f035-4886-a2c3-579fc6482430-plugin-serving-cert podName:5b97797a-f035-4886-a2c3-579fc6482430 nodeName:}" failed. No retries permitted until 2026-07-16 10:39:36.921129704 +0000 UTC m=+431.600394668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/5b97797a-f035-4886-a2c3-579fc6482430-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-ldstt" (UID: "5b97797a-f035-4886-a2c3-579fc6482430") : secret "plugin-serving-cert" not found Jul 16 10:39:36.421517 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.421502 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5b97797a-f035-4886-a2c3-579fc6482430-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-ldstt\" (UID: \"5b97797a-f035-4886-a2c3-579fc6482430\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:36.447122 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.447064 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9vdk\" (UniqueName: \"kubernetes.io/projected/5b97797a-f035-4886-a2c3-579fc6482430-kube-api-access-n9vdk\") pod \"kuadrant-console-plugin-66b5774985-ldstt\" (UID: \"5b97797a-f035-4886-a2c3-579fc6482430\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:36.924922 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.924885 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b97797a-f035-4886-a2c3-579fc6482430-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-ldstt\" (UID: \"5b97797a-f035-4886-a2c3-579fc6482430\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:36.927324 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:36.927300 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b97797a-f035-4886-a2c3-579fc6482430-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-ldstt\" (UID: \"5b97797a-f035-4886-a2c3-579fc6482430\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:37.106739 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:37.106705 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" Jul 16 10:39:37.267966 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:37.267944 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt"] Jul 16 10:39:37.270060 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:37.270032 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b97797a_f035_4886_a2c3_579fc6482430.slice/crio-21ce894469eed4945a75bcc78e0ae91eee529271228c4eb4736cd6d812debefe WatchSource:0}: Error finding container 21ce894469eed4945a75bcc78e0ae91eee529271228c4eb4736cd6d812debefe: Status 404 returned error can't find the container with id 21ce894469eed4945a75bcc78e0ae91eee529271228c4eb4736cd6d812debefe Jul 16 10:39:37.667014 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:37.666980 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" event={"ID":"5b97797a-f035-4886-a2c3-579fc6482430","Type":"ContainerStarted","Data":"21ce894469eed4945a75bcc78e0ae91eee529271228c4eb4736cd6d812debefe"} Jul 16 10:39:44.653663 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:44.653625 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:46.547568 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.547532 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv"] Jul 16 10:39:46.548293 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.547786 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" containerName="manager" containerID="cri-o://74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347" gracePeriod=2 Jul 16 10:39:46.550780 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.550753 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv"] Jul 16 10:39:46.559808 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:46.559756 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:46.564964 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.564940 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx"] Jul 16 10:39:46.565197 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.565173 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" podUID="73f7b472-2485-4225-b915-d4414e2816df" containerName="manager" containerID="cri-o://fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad" gracePeriod=2 Jul 16 10:39:46.567765 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:46.567730 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:46.581452 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.581419 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx"] Jul 16 10:39:46.600104 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.600073 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq"] Jul 16 10:39:46.600511 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.600486 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" containerName="manager" Jul 16 10:39:46.600511 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.600513 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" containerName="manager" Jul 16 10:39:46.600734 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.600533 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73f7b472-2485-4225-b915-d4414e2816df" containerName="manager" Jul 16 10:39:46.600734 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.600542 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="73f7b472-2485-4225-b915-d4414e2816df" containerName="manager" Jul 16 10:39:46.600734 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.600669 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="73f7b472-2485-4225-b915-d4414e2816df" containerName="manager" Jul 16 10:39:46.600734 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.600703 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" containerName="manager" Jul 16 10:39:46.607485 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.607460 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:39:46.611939 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.611912 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4"] Jul 16 10:39:46.615877 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.615858 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4" Jul 16 10:39:46.630098 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.630002 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq"] Jul 16 10:39:46.658264 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.658225 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4"] Jul 16 10:39:46.658913 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:46.658878 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:46.662709 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:46.662666 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rpzpx\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="73f7b472-2485-4225-b915-d4414e2816df" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:46.664079 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.664055 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s6ndq" Jul 16 10:39:46.704203 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:46.704170 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:46.706845 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:46.706799 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rpzpx\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="73f7b472-2485-4225-b915-d4414e2816df" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:46.714643 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.714610 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8jbf\" (UniqueName: \"kubernetes.io/projected/686552d5-633e-4951-b588-a994b6e8becf-kube-api-access-r8jbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6gqwq\" (UID: \"686552d5-633e-4951-b588-a994b6e8becf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:39:46.714804 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.714721 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/686552d5-633e-4951-b588-a994b6e8becf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6gqwq\" (UID: \"686552d5-633e-4951-b588-a994b6e8becf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:39:46.714804 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.714757 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n85d2\" (UniqueName: \"kubernetes.io/projected/6049c64a-a51d-4353-b311-dd4f62601d61-kube-api-access-n85d2\") pod \"limitador-operator-controller-manager-859d7b78bd-9w7v4\" (UID: \"6049c64a-a51d-4353-b311-dd4f62601d61\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4" Jul 16 10:39:46.760643 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:46.760601 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:46.765856 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:46.765818 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rpzpx\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="73f7b472-2485-4225-b915-d4414e2816df" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:46.815238 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.815195 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/686552d5-633e-4951-b588-a994b6e8becf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6gqwq\" (UID: \"686552d5-633e-4951-b588-a994b6e8becf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:39:46.815400 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.815247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n85d2\" (UniqueName: \"kubernetes.io/projected/6049c64a-a51d-4353-b311-dd4f62601d61-kube-api-access-n85d2\") pod \"limitador-operator-controller-manager-859d7b78bd-9w7v4\" (UID: \"6049c64a-a51d-4353-b311-dd4f62601d61\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4" Jul 16 10:39:46.815400 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.815332 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8jbf\" (UniqueName: \"kubernetes.io/projected/686552d5-633e-4951-b588-a994b6e8becf-kube-api-access-r8jbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6gqwq\" (UID: \"686552d5-633e-4951-b588-a994b6e8becf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:39:46.815590 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.815565 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/686552d5-633e-4951-b588-a994b6e8becf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6gqwq\" (UID: \"686552d5-633e-4951-b588-a994b6e8becf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:39:46.826539 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.826511 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8jbf\" (UniqueName: \"kubernetes.io/projected/686552d5-633e-4951-b588-a994b6e8becf-kube-api-access-r8jbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6gqwq\" (UID: \"686552d5-633e-4951-b588-a994b6e8becf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:39:46.827832 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.827804 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n85d2\" (UniqueName: \"kubernetes.io/projected/6049c64a-a51d-4353-b311-dd4f62601d61-kube-api-access-n85d2\") pod \"limitador-operator-controller-manager-859d7b78bd-9w7v4\" (UID: \"6049c64a-a51d-4353-b311-dd4f62601d61\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4" Jul 16 10:39:46.979974 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.979894 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:39:46.987910 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:46.987878 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4" Jul 16 10:39:58.153586 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.153552 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:58.157600 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:58.157565 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:58.166719 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.166677 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:58.177451 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:58.177413 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:58.180103 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:58.180072 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rpzpx\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="73f7b472-2485-4225-b915-d4414e2816df" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:58.210719 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.210660 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4"] Jul 16 10:39:58.212190 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:58.212098 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6049c64a_a51d_4353_b311_dd4f62601d61.slice/crio-001ec52a5c278532cd43a30038d5cfec0662def4cc2844d30006d34135d3d673 WatchSource:0}: Error finding container 001ec52a5c278532cd43a30038d5cfec0662def4cc2844d30006d34135d3d673: Status 404 returned error can't find the container with id 001ec52a5c278532cd43a30038d5cfec0662def4cc2844d30006d34135d3d673 Jul 16 10:39:58.213765 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.213740 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq"] Jul 16 10:39:58.214473 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:39:58.214449 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod686552d5_633e_4951_b588_a994b6e8becf.slice/crio-1d71a64d1a30c7245aa3dcb09497e3cc7f8a47fe6a2fe83123c417abbc794ba8 WatchSource:0}: Error finding container 1d71a64d1a30c7245aa3dcb09497e3cc7f8a47fe6a2fe83123c417abbc794ba8: Status 404 returned error can't find the container with id 1d71a64d1a30c7245aa3dcb09497e3cc7f8a47fe6a2fe83123c417abbc794ba8 Jul 16 10:39:58.215958 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.215938 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km4ps\" (UniqueName: \"kubernetes.io/projected/b0cc54cc-14db-4a51-a565-863cd7fae54d-kube-api-access-km4ps\") pod \"b0cc54cc-14db-4a51-a565-863cd7fae54d\" (UID: \"b0cc54cc-14db-4a51-a565-863cd7fae54d\") " Jul 16 10:39:58.216038 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.216024 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0cc54cc-14db-4a51-a565-863cd7fae54d-extensions-socket-volume\") pod \"b0cc54cc-14db-4a51-a565-863cd7fae54d\" (UID: \"b0cc54cc-14db-4a51-a565-863cd7fae54d\") " Jul 16 10:39:58.216074 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.216063 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5brw\" (UniqueName: \"kubernetes.io/projected/73f7b472-2485-4225-b915-d4414e2816df-kube-api-access-g5brw\") pod \"73f7b472-2485-4225-b915-d4414e2816df\" (UID: \"73f7b472-2485-4225-b915-d4414e2816df\") " Jul 16 10:39:58.216631 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.216606 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0cc54cc-14db-4a51-a565-863cd7fae54d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b0cc54cc-14db-4a51-a565-863cd7fae54d" (UID: "b0cc54cc-14db-4a51-a565-863cd7fae54d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:39:58.218107 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.218079 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0cc54cc-14db-4a51-a565-863cd7fae54d-kube-api-access-km4ps" (OuterVolumeSpecName: "kube-api-access-km4ps") pod "b0cc54cc-14db-4a51-a565-863cd7fae54d" (UID: "b0cc54cc-14db-4a51-a565-863cd7fae54d"). InnerVolumeSpecName "kube-api-access-km4ps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:39:58.218284 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.218254 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73f7b472-2485-4225-b915-d4414e2816df-kube-api-access-g5brw" (OuterVolumeSpecName: "kube-api-access-g5brw") pod "73f7b472-2485-4225-b915-d4414e2816df" (UID: "73f7b472-2485-4225-b915-d4414e2816df"). InnerVolumeSpecName "kube-api-access-g5brw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:39:58.317512 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.317475 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km4ps\" (UniqueName: \"kubernetes.io/projected/b0cc54cc-14db-4a51-a565-863cd7fae54d-kube-api-access-km4ps\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:58.317512 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.317503 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0cc54cc-14db-4a51-a565-863cd7fae54d-extensions-socket-volume\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:58.317512 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.317514 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5brw\" (UniqueName: \"kubernetes.io/projected/73f7b472-2485-4225-b915-d4414e2816df-kube-api-access-g5brw\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:39:58.798798 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.798761 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" event={"ID":"686552d5-633e-4951-b588-a994b6e8becf","Type":"ContainerStarted","Data":"a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b"} Jul 16 10:39:58.798988 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.798812 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" event={"ID":"686552d5-633e-4951-b588-a994b6e8becf","Type":"ContainerStarted","Data":"1d71a64d1a30c7245aa3dcb09497e3cc7f8a47fe6a2fe83123c417abbc794ba8"} Jul 16 10:39:58.798988 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.798856 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:39:58.800362 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.800337 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4" event={"ID":"6049c64a-a51d-4353-b311-dd4f62601d61","Type":"ContainerStarted","Data":"49dc48216ba77f1e60dc02aea1e3e1fd03dc735218cebd2dd683f3b172919e53"} Jul 16 10:39:58.800493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.800372 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4" event={"ID":"6049c64a-a51d-4353-b311-dd4f62601d61","Type":"ContainerStarted","Data":"001ec52a5c278532cd43a30038d5cfec0662def4cc2844d30006d34135d3d673"} Jul 16 10:39:58.800493 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.800405 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4" Jul 16 10:39:58.801563 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.801543 2581 generic.go:356] "Generic (PLEG): container finished" podID="b0cc54cc-14db-4a51-a565-863cd7fae54d" containerID="74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347" exitCode=0 Jul 16 10:39:58.801630 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.801577 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:58.801670 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.801580 2581 scope.go:117] "RemoveContainer" containerID="74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347" Jul 16 10:39:58.802819 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.802803 2581 generic.go:356] "Generic (PLEG): container finished" podID="73f7b472-2485-4225-b915-d4414e2816df" containerID="fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad" exitCode=0 Jul 16 10:39:58.802878 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.802845 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:58.812602 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.812576 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" event={"ID":"5b97797a-f035-4886-a2c3-579fc6482430","Type":"ContainerStarted","Data":"6964a8f48ad8975c3671a1488b423dcd4b6850021654a1f02c0945dd2eaffc2d"} Jul 16 10:39:58.816484 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.816469 2581 scope.go:117] "RemoveContainer" containerID="74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347" Jul 16 10:39:58.816734 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:58.816717 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347\": container with ID starting with 74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347 not found: ID does not exist" containerID="74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347" Jul 16 10:39:58.816786 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.816741 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347"} err="failed to get container status \"74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347\": rpc error: code = NotFound desc = could not find container \"74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347\": container with ID starting with 74e0a9799dbe43e0a0a3b4a21fd23f42a6254682182261081bc9e17a1f918347 not found: ID does not exist" Jul 16 10:39:58.816786 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.816757 2581 scope.go:117] "RemoveContainer" containerID="fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad" Jul 16 10:39:58.824463 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.824447 2581 scope.go:117] "RemoveContainer" containerID="fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad" Jul 16 10:39:58.824732 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:58.824713 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad\": container with ID starting with fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad not found: ID does not exist" containerID="fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad" Jul 16 10:39:58.824785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.824740 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad"} err="failed to get container status \"fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad\": rpc error: code = NotFound desc = could not find container \"fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad\": container with ID starting with fe7cb07da80c21e1badd98fc4a53d5b4fc72cb141ddabc553e78831cf751e3ad not found: ID does not exist" Jul 16 10:39:58.837212 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:58.837175 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:58.837364 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.837213 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" podStartSLOduration=12.837193847 podStartE2EDuration="12.837193847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:39:58.834670248 +0000 UTC m=+453.513935227" watchObservedRunningTime="2026-07-16 10:39:58.837193847 +0000 UTC m=+453.516458827" Jul 16 10:39:58.840492 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:58.840457 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rpzpx\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="73f7b472-2485-4225-b915-d4414e2816df" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:58.876952 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.876902 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4" podStartSLOduration=12.876886579 podStartE2EDuration="12.876886579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:39:58.875999643 +0000 UTC m=+453.555264621" watchObservedRunningTime="2026-07-16 10:39:58.876886579 +0000 UTC m=+453.556151557" Jul 16 10:39:58.878416 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:58.878385 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tl2xv\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tl2xv" Jul 16 10:39:58.881054 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:39:58.881036 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rpzpx\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="73f7b472-2485-4225-b915-d4414e2816df" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rpzpx" Jul 16 10:39:58.904377 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:58.904319 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-ldstt" podStartSLOduration=2.118363437 podStartE2EDuration="22.904303664s" totalImagesPullingTime="20.785940227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:39:58.90145491 +0000 UTC m=+453.580719891" watchObservedRunningTime="2026-07-16 10:39:58.904303664 +0000 UTC m=+453.583568643" Jul 16 10:39:59.806020 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:59.805988 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73f7b472-2485-4225-b915-d4414e2816df" path="/var/lib/kubelet/pods/73f7b472-2485-4225-b915-d4414e2816df/volumes" Jul 16 10:39:59.806388 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:39:59.806328 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0cc54cc-14db-4a51-a565-863cd7fae54d" path="/var/lib/kubelet/pods/b0cc54cc-14db-4a51-a565-863cd7fae54d/volumes" Jul 16 10:40:03.741254 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:03.741218 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq"] Jul 16 10:40:03.741780 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:03.741430 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" podUID="686552d5-633e-4951-b588-a994b6e8becf" containerName="manager" containerID="cri-o://a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b" gracePeriod=10 Jul 16 10:40:03.743300 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:03.743280 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:40:03.992075 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:03.992022 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:40:04.068443 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.068408 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8jbf\" (UniqueName: \"kubernetes.io/projected/686552d5-633e-4951-b588-a994b6e8becf-kube-api-access-r8jbf\") pod \"686552d5-633e-4951-b588-a994b6e8becf\" (UID: \"686552d5-633e-4951-b588-a994b6e8becf\") " Jul 16 10:40:04.068581 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.068450 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/686552d5-633e-4951-b588-a994b6e8becf-extensions-socket-volume\") pod \"686552d5-633e-4951-b588-a994b6e8becf\" (UID: \"686552d5-633e-4951-b588-a994b6e8becf\") " Jul 16 10:40:04.068919 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.068896 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686552d5-633e-4951-b588-a994b6e8becf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "686552d5-633e-4951-b588-a994b6e8becf" (UID: "686552d5-633e-4951-b588-a994b6e8becf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 10:40:04.070435 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.070413 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/686552d5-633e-4951-b588-a994b6e8becf-kube-api-access-r8jbf" (OuterVolumeSpecName: "kube-api-access-r8jbf") pod "686552d5-633e-4951-b588-a994b6e8becf" (UID: "686552d5-633e-4951-b588-a994b6e8becf"). InnerVolumeSpecName "kube-api-access-r8jbf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:40:04.174806 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.170331 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8jbf\" (UniqueName: \"kubernetes.io/projected/686552d5-633e-4951-b588-a994b6e8becf-kube-api-access-r8jbf\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:40:04.174806 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.170370 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/686552d5-633e-4951-b588-a994b6e8becf-extensions-socket-volume\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:40:04.845006 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.844973 2581 generic.go:356] "Generic (PLEG): container finished" podID="686552d5-633e-4951-b588-a994b6e8becf" containerID="a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b" exitCode=0 Jul 16 10:40:04.845427 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.845031 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" Jul 16 10:40:04.845427 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.845051 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" event={"ID":"686552d5-633e-4951-b588-a994b6e8becf","Type":"ContainerDied","Data":"a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b"} Jul 16 10:40:04.845427 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.845090 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq" event={"ID":"686552d5-633e-4951-b588-a994b6e8becf","Type":"ContainerDied","Data":"1d71a64d1a30c7245aa3dcb09497e3cc7f8a47fe6a2fe83123c417abbc794ba8"} Jul 16 10:40:04.845427 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.845110 2581 scope.go:117] "RemoveContainer" containerID="a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b" Jul 16 10:40:04.854586 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.854571 2581 scope.go:117] "RemoveContainer" containerID="a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b" Jul 16 10:40:04.854896 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:40:04.854875 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b\": container with ID starting with a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b not found: ID does not exist" containerID="a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b" Jul 16 10:40:04.854955 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.854905 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b"} err="failed to get container status \"a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b\": rpc error: code = NotFound desc = could not find container \"a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b\": container with ID starting with a01542b2df2d225ea0c048100b87e08e3b44413f577fd074ce56bea42b11069b not found: ID does not exist" Jul 16 10:40:04.879265 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.879245 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq"] Jul 16 10:40:04.884888 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:04.884870 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6gqwq"] Jul 16 10:40:05.806772 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:05.806739 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="686552d5-633e-4951-b588-a994b6e8becf" path="/var/lib/kubelet/pods/686552d5-633e-4951-b588-a994b6e8becf/volumes" Jul 16 10:40:09.821333 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:09.821285 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9w7v4" Jul 16 10:40:20.247206 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.247176 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq"] Jul 16 10:40:20.247801 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.247545 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="686552d5-633e-4951-b588-a994b6e8becf" containerName="manager" Jul 16 10:40:20.247801 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.247560 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="686552d5-633e-4951-b588-a994b6e8becf" containerName="manager" Jul 16 10:40:20.247801 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.247626 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="686552d5-633e-4951-b588-a994b6e8becf" containerName="manager" Jul 16 10:40:20.337772 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.337723 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq"] Jul 16 10:40:20.337969 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.337817 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.340824 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.340803 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 10:40:20.340824 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.340805 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dj9xz\"" Jul 16 10:40:20.408488 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.408448 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vgrn\" (UniqueName: \"kubernetes.io/projected/e6bcaaaa-9f7e-4c89-8c94-c66486302286-kube-api-access-2vgrn\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.408488 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.408498 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.408779 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.408595 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.408779 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.408675 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.408779 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.408723 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.408779 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.408776 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.408961 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.408799 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.408961 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.408824 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.408961 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.408854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.510326 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.510240 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.510326 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.510283 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.510326 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.510310 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.510621 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.510341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.510621 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.510370 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vgrn\" (UniqueName: \"kubernetes.io/projected/e6bcaaaa-9f7e-4c89-8c94-c66486302286-kube-api-access-2vgrn\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.510621 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.510399 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.510621 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.510437 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.510621 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.510501 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.510621 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.510532 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.511002 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.510809 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.511002 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.510943 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.511094 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.511048 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.511191 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.511171 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.511230 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.511179 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.512811 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.512794 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.513132 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.513103 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.520824 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.520793 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vgrn\" (UniqueName: \"kubernetes.io/projected/e6bcaaaa-9f7e-4c89-8c94-c66486302286-kube-api-access-2vgrn\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.521535 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.521509 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6bcaaaa-9f7e-4c89-8c94-c66486302286-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-g82mq\" (UID: \"e6bcaaaa-9f7e-4c89-8c94-c66486302286\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.648561 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.648528 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:20.861895 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.861862 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq"] Jul 16 10:40:20.865090 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:40:20.865062 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6bcaaaa_9f7e_4c89_8c94_c66486302286.slice/crio-170f1bea5c8cf8307b94d8dfb3294e8facd6b5c820a12ce52c274046d19c5ba3 WatchSource:0}: Error finding container 170f1bea5c8cf8307b94d8dfb3294e8facd6b5c820a12ce52c274046d19c5ba3: Status 404 returned error can't find the container with id 170f1bea5c8cf8307b94d8dfb3294e8facd6b5c820a12ce52c274046d19c5ba3 Jul 16 10:40:20.932575 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:20.932540 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" event={"ID":"e6bcaaaa-9f7e-4c89-8c94-c66486302286","Type":"ContainerStarted","Data":"170f1bea5c8cf8307b94d8dfb3294e8facd6b5c820a12ce52c274046d19c5ba3"} Jul 16 10:40:25.615887 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:25.615853 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 10:40:25.616133 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:25.615928 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 10:40:25.616133 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:25.615958 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 10:40:25.964211 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:25.964125 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" event={"ID":"e6bcaaaa-9f7e-4c89-8c94-c66486302286","Type":"ContainerStarted","Data":"402c1b5bc9a289cdc64053366c74bb3a11a3a44ee469383ecb73e80067ddb652"} Jul 16 10:40:25.991832 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:25.991777 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" podStartSLOduration=1.243287715 podStartE2EDuration="5.991761293s" totalImagesPullingTime="4.748473578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:40:25.989776727 +0000 UTC m=+480.669041713" watchObservedRunningTime="2026-07-16 10:40:25.991761293 +0000 UTC m=+480.671026270" Jul 16 10:40:26.649763 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:26.649726 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:26.654817 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:26.654790 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:26.968917 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:26.968842 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:26.969744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:26.969722 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:40:27.093287 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.093253 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vk5dn"] Jul 16 10:40:27.098550 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.098528 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:27.101454 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.101428 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 10:40:27.109260 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.109234 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vk5dn"] Jul 16 10:40:27.169973 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.169925 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-config-file\") pod \"limitador-limitador-64c94fd64f-vk5dn\" (UID: \"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:27.170152 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.169988 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lkdn\" (UniqueName: \"kubernetes.io/projected/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-kube-api-access-8lkdn\") pod \"limitador-limitador-64c94fd64f-vk5dn\" (UID: \"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:27.180062 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.180026 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vk5dn"] Jul 16 10:40:27.271101 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.270998 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-config-file\") pod \"limitador-limitador-64c94fd64f-vk5dn\" (UID: \"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:27.271101 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.271058 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lkdn\" (UniqueName: \"kubernetes.io/projected/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-kube-api-access-8lkdn\") pod \"limitador-limitador-64c94fd64f-vk5dn\" (UID: \"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:27.271670 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.271644 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-config-file\") pod \"limitador-limitador-64c94fd64f-vk5dn\" (UID: \"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:27.281561 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.281535 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lkdn\" (UniqueName: \"kubernetes.io/projected/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-kube-api-access-8lkdn\") pod \"limitador-limitador-64c94fd64f-vk5dn\" (UID: \"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:27.409589 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.409557 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:27.460354 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.460318 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:40:27.468617 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.467767 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" Jul 16 10:40:27.483832 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.483769 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:40:27.522929 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.522312 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:40:27.564080 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.564048 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vk5dn"] Jul 16 10:40:27.565143 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:40:27.565118 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c9a29a_076d_4826_a5d9_9dfd57ccabcb.slice/crio-f1938f92df0ccf84d95aa5a9113ded36dcd80402760ed2c836c5f24ba5b823e8 WatchSource:0}: Error finding container f1938f92df0ccf84d95aa5a9113ded36dcd80402760ed2c836c5f24ba5b823e8: Status 404 returned error can't find the container with id f1938f92df0ccf84d95aa5a9113ded36dcd80402760ed2c836c5f24ba5b823e8 Jul 16 10:40:27.573148 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.573122 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3bfad0fc-6892-4b7d-842d-d8c597ce1505-config-file\") pod \"limitador-limitador-54d6d988d5-xrfw2\" (UID: \"3bfad0fc-6892-4b7d-842d-d8c597ce1505\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" Jul 16 10:40:27.573250 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.573200 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b7qr\" (UniqueName: \"kubernetes.io/projected/3bfad0fc-6892-4b7d-842d-d8c597ce1505-kube-api-access-7b7qr\") pod \"limitador-limitador-54d6d988d5-xrfw2\" (UID: \"3bfad0fc-6892-4b7d-842d-d8c597ce1505\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" Jul 16 10:40:27.674457 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.674422 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3bfad0fc-6892-4b7d-842d-d8c597ce1505-config-file\") pod \"limitador-limitador-54d6d988d5-xrfw2\" (UID: \"3bfad0fc-6892-4b7d-842d-d8c597ce1505\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" Jul 16 10:40:27.674893 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.674489 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b7qr\" (UniqueName: \"kubernetes.io/projected/3bfad0fc-6892-4b7d-842d-d8c597ce1505-kube-api-access-7b7qr\") pod \"limitador-limitador-54d6d988d5-xrfw2\" (UID: \"3bfad0fc-6892-4b7d-842d-d8c597ce1505\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" Jul 16 10:40:27.675086 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.675066 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3bfad0fc-6892-4b7d-842d-d8c597ce1505-config-file\") pod \"limitador-limitador-54d6d988d5-xrfw2\" (UID: \"3bfad0fc-6892-4b7d-842d-d8c597ce1505\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" Jul 16 10:40:27.685927 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.685898 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b7qr\" (UniqueName: \"kubernetes.io/projected/3bfad0fc-6892-4b7d-842d-d8c597ce1505-kube-api-access-7b7qr\") pod \"limitador-limitador-54d6d988d5-xrfw2\" (UID: \"3bfad0fc-6892-4b7d-842d-d8c597ce1505\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" Jul 16 10:40:27.785920 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.785826 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" Jul 16 10:40:27.924770 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.924743 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:40:27.925843 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:40:27.925819 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bfad0fc_6892_4b7d_842d_d8c597ce1505.slice/crio-51d201adedfc66c92f912775e950b7e1a99892c2f859d580f7ee6bd5e0c8b822 WatchSource:0}: Error finding container 51d201adedfc66c92f912775e950b7e1a99892c2f859d580f7ee6bd5e0c8b822: Status 404 returned error can't find the container with id 51d201adedfc66c92f912775e950b7e1a99892c2f859d580f7ee6bd5e0c8b822 Jul 16 10:40:27.975130 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.975086 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" event={"ID":"3bfad0fc-6892-4b7d-842d-d8c597ce1505","Type":"ContainerStarted","Data":"51d201adedfc66c92f912775e950b7e1a99892c2f859d580f7ee6bd5e0c8b822"} Jul 16 10:40:27.976415 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:27.976386 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" event={"ID":"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb","Type":"ContainerStarted","Data":"f1938f92df0ccf84d95aa5a9113ded36dcd80402760ed2c836c5f24ba5b823e8"} Jul 16 10:40:30.996838 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:30.996743 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" event={"ID":"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb","Type":"ContainerStarted","Data":"2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891"} Jul 16 10:40:30.997477 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:30.997034 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:30.998516 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:30.998490 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" event={"ID":"3bfad0fc-6892-4b7d-842d-d8c597ce1505","Type":"ContainerStarted","Data":"6081f904a3f6901b0c6f6ce55f179327a279c73bf6d545422adc0dacd8e8c68e"} Jul 16 10:40:30.998659 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:30.998583 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" Jul 16 10:40:31.028645 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:31.028587 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" podStartSLOduration=0.915087816 podStartE2EDuration="4.028571081s" totalImagesPullingTime="3.113483265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:40:31.027539732 +0000 UTC m=+485.706804711" watchObservedRunningTime="2026-07-16 10:40:31.028571081 +0000 UTC m=+485.707836060" Jul 16 10:40:31.062005 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:31.061941 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" podStartSLOduration=1.29944938 podStartE2EDuration="4.061919925s" totalImagesPullingTime="2.762470545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:40:31.057862991 +0000 UTC m=+485.737127972" watchObservedRunningTime="2026-07-16 10:40:31.061919925 +0000 UTC m=+485.741184905" Jul 16 10:40:42.005159 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:42.005133 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:42.005548 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:42.005311 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrfw2" Jul 16 10:40:42.101809 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:42.101775 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vk5dn"] Jul 16 10:40:42.101983 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:42.101959 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" podUID="c5c9a29a-076d-4826-a5d9-9dfd57ccabcb" containerName="limitador" containerID="cri-o://2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891" gracePeriod=30 Jul 16 10:40:42.645504 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:42.645471 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:42.710320 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:42.710212 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-config-file\") pod \"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb\" (UID: \"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb\") " Jul 16 10:40:42.710320 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:42.710282 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lkdn\" (UniqueName: \"kubernetes.io/projected/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-kube-api-access-8lkdn\") pod \"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb\" (UID: \"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb\") " Jul 16 10:40:42.710603 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:42.710581 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-config-file" (OuterVolumeSpecName: "config-file") pod "c5c9a29a-076d-4826-a5d9-9dfd57ccabcb" (UID: "c5c9a29a-076d-4826-a5d9-9dfd57ccabcb"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 10:40:42.712129 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:42.712100 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-kube-api-access-8lkdn" (OuterVolumeSpecName: "kube-api-access-8lkdn") pod "c5c9a29a-076d-4826-a5d9-9dfd57ccabcb" (UID: "c5c9a29a-076d-4826-a5d9-9dfd57ccabcb"). InnerVolumeSpecName "kube-api-access-8lkdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:40:42.811509 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:42.811480 2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-config-file\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:40:42.811509 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:42.811503 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lkdn\" (UniqueName: \"kubernetes.io/projected/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb-kube-api-access-8lkdn\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:40:43.061836 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:43.061744 2581 generic.go:356] "Generic (PLEG): container finished" podID="c5c9a29a-076d-4826-a5d9-9dfd57ccabcb" containerID="2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891" exitCode=0 Jul 16 10:40:43.061836 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:43.061789 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" event={"ID":"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb","Type":"ContainerDied","Data":"2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891"} Jul 16 10:40:43.061836 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:43.061799 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" Jul 16 10:40:43.062311 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:43.061840 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vk5dn" event={"ID":"c5c9a29a-076d-4826-a5d9-9dfd57ccabcb","Type":"ContainerDied","Data":"f1938f92df0ccf84d95aa5a9113ded36dcd80402760ed2c836c5f24ba5b823e8"} Jul 16 10:40:43.062311 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:43.061869 2581 scope.go:117] "RemoveContainer" containerID="2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891" Jul 16 10:40:43.071011 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:43.070989 2581 scope.go:117] "RemoveContainer" containerID="2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891" Jul 16 10:40:43.071278 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:40:43.071261 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891\": container with ID starting with 2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891 not found: ID does not exist" containerID="2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891" Jul 16 10:40:43.071345 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:43.071290 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891"} err="failed to get container status \"2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891\": rpc error: code = NotFound desc = could not find container \"2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891\": container with ID starting with 2d4dcf90ddd896a9fbf984f50d030ba3b988688455b88d20c6bcd5fdfe033891 not found: ID does not exist" Jul 16 10:40:43.087050 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:43.087023 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vk5dn"] Jul 16 10:40:43.092161 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:43.092138 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vk5dn"] Jul 16 10:40:43.806518 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:40:43.806488 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5c9a29a-076d-4826-a5d9-9dfd57ccabcb" path="/var/lib/kubelet/pods/c5c9a29a-076d-4826-a5d9-9dfd57ccabcb/volumes" Jul 16 10:41:00.551224 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.551179 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l9n8t"] Jul 16 10:41:00.551714 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.551684 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5c9a29a-076d-4826-a5d9-9dfd57ccabcb" containerName="limitador" Jul 16 10:41:00.551764 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.551717 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5c9a29a-076d-4826-a5d9-9dfd57ccabcb" containerName="limitador" Jul 16 10:41:00.551819 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.551808 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5c9a29a-076d-4826-a5d9-9dfd57ccabcb" containerName="limitador" Jul 16 10:41:00.560203 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.560180 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-l9n8t" Jul 16 10:41:00.563423 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.563394 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l9n8t"] Jul 16 10:41:00.563569 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.563472 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fd6bn\"" Jul 16 10:41:00.667049 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.667015 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t8tb\" (UniqueName: \"kubernetes.io/projected/5baea508-dd5d-4b31-b7a7-0812dce80cf0-kube-api-access-4t8tb\") pod \"authorino-5d4b67c446-l9n8t\" (UID: \"5baea508-dd5d-4b31-b7a7-0812dce80cf0\") " pod="kuadrant-system/authorino-5d4b67c446-l9n8t" Jul 16 10:41:00.768312 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.768272 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t8tb\" (UniqueName: \"kubernetes.io/projected/5baea508-dd5d-4b31-b7a7-0812dce80cf0-kube-api-access-4t8tb\") pod \"authorino-5d4b67c446-l9n8t\" (UID: \"5baea508-dd5d-4b31-b7a7-0812dce80cf0\") " pod="kuadrant-system/authorino-5d4b67c446-l9n8t" Jul 16 10:41:00.780298 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.780266 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t8tb\" (UniqueName: \"kubernetes.io/projected/5baea508-dd5d-4b31-b7a7-0812dce80cf0-kube-api-access-4t8tb\") pod \"authorino-5d4b67c446-l9n8t\" (UID: \"5baea508-dd5d-4b31-b7a7-0812dce80cf0\") " pod="kuadrant-system/authorino-5d4b67c446-l9n8t" Jul 16 10:41:00.810897 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.810811 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l9n8t"] Jul 16 10:41:00.811081 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.811069 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-l9n8t" Jul 16 10:41:00.851578 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.851544 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d8476c4d5-w5bt8"] Jul 16 10:41:00.856774 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.856751 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" Jul 16 10:41:00.871094 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.871071 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d8476c4d5-w5bt8"] Jul 16 10:41:00.956132 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.955007 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l9n8t"] Jul 16 10:41:00.970808 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:00.970773 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mhbx\" (UniqueName: \"kubernetes.io/projected/e17b881e-42d4-4582-b9f4-6a35075a354b-kube-api-access-5mhbx\") pod \"authorino-6d8476c4d5-w5bt8\" (UID: \"e17b881e-42d4-4582-b9f4-6a35075a354b\") " pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" Jul 16 10:41:01.071786 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:01.071748 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mhbx\" (UniqueName: \"kubernetes.io/projected/e17b881e-42d4-4582-b9f4-6a35075a354b-kube-api-access-5mhbx\") pod \"authorino-6d8476c4d5-w5bt8\" (UID: \"e17b881e-42d4-4582-b9f4-6a35075a354b\") " pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" Jul 16 10:41:01.083743 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:01.083710 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mhbx\" (UniqueName: \"kubernetes.io/projected/e17b881e-42d4-4582-b9f4-6a35075a354b-kube-api-access-5mhbx\") pod \"authorino-6d8476c4d5-w5bt8\" (UID: \"e17b881e-42d4-4582-b9f4-6a35075a354b\") " pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" Jul 16 10:41:01.154760 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:01.154722 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-l9n8t" event={"ID":"5baea508-dd5d-4b31-b7a7-0812dce80cf0","Type":"ContainerStarted","Data":"34f7a4885c246910f1b53d686db3b7dd4be5762d0baa3fa0c2fd0808fbfb7513"} Jul 16 10:41:01.168963 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:01.168934 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" Jul 16 10:41:01.186744 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:01.186719 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d8476c4d5-w5bt8"] Jul 16 10:41:01.314681 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:01.314652 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d8476c4d5-w5bt8"] Jul 16 10:41:01.316930 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:41:01.316902 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode17b881e_42d4_4582_b9f4_6a35075a354b.slice/crio-ba69fe4d20a418c45b2633b08bb47f5acd7f0ed8df0d125a45770c965b026fda WatchSource:0}: Error finding container ba69fe4d20a418c45b2633b08bb47f5acd7f0ed8df0d125a45770c965b026fda: Status 404 returned error can't find the container with id ba69fe4d20a418c45b2633b08bb47f5acd7f0ed8df0d125a45770c965b026fda Jul 16 10:41:02.168468 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:02.168425 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" event={"ID":"e17b881e-42d4-4582-b9f4-6a35075a354b","Type":"ContainerStarted","Data":"ba69fe4d20a418c45b2633b08bb47f5acd7f0ed8df0d125a45770c965b026fda"} Jul 16 10:41:05.190270 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.190229 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-l9n8t" event={"ID":"5baea508-dd5d-4b31-b7a7-0812dce80cf0","Type":"ContainerStarted","Data":"fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045"} Jul 16 10:41:05.190771 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.190287 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-l9n8t" podUID="5baea508-dd5d-4b31-b7a7-0812dce80cf0" containerName="authorino" containerID="cri-o://fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045" gracePeriod=30 Jul 16 10:41:05.191766 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.191735 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" event={"ID":"e17b881e-42d4-4582-b9f4-6a35075a354b","Type":"ContainerStarted","Data":"3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf"} Jul 16 10:41:05.191878 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.191805 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" podUID="e17b881e-42d4-4582-b9f4-6a35075a354b" containerName="authorino" containerID="cri-o://3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf" gracePeriod=30 Jul 16 10:41:05.209100 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.209059 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-l9n8t" podStartSLOduration=1.8831828879999999 podStartE2EDuration="5.209042468s" totalImagesPullingTime="3.32585958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:41:05.208341647 +0000 UTC m=+519.887606628" watchObservedRunningTime="2026-07-16 10:41:05.209042468 +0000 UTC m=+519.888307446" Jul 16 10:41:05.228163 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.228118 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" podStartSLOduration=2.251378104 podStartE2EDuration="5.228102657s" totalImagesPullingTime="2.976724553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:41:05.225946416 +0000 UTC m=+519.905211397" watchObservedRunningTime="2026-07-16 10:41:05.228102657 +0000 UTC m=+519.907367636" Jul 16 10:41:05.437593 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.437564 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" Jul 16 10:41:05.475542 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.475518 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-l9n8t" Jul 16 10:41:05.517670 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.517641 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mhbx\" (UniqueName: \"kubernetes.io/projected/e17b881e-42d4-4582-b9f4-6a35075a354b-kube-api-access-5mhbx\") pod \"e17b881e-42d4-4582-b9f4-6a35075a354b\" (UID: \"e17b881e-42d4-4582-b9f4-6a35075a354b\") " Jul 16 10:41:05.519738 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.519711 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e17b881e-42d4-4582-b9f4-6a35075a354b-kube-api-access-5mhbx" (OuterVolumeSpecName: "kube-api-access-5mhbx") pod "e17b881e-42d4-4582-b9f4-6a35075a354b" (UID: "e17b881e-42d4-4582-b9f4-6a35075a354b"). InnerVolumeSpecName "kube-api-access-5mhbx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:41:05.618439 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.618395 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t8tb\" (UniqueName: \"kubernetes.io/projected/5baea508-dd5d-4b31-b7a7-0812dce80cf0-kube-api-access-4t8tb\") pod \"5baea508-dd5d-4b31-b7a7-0812dce80cf0\" (UID: \"5baea508-dd5d-4b31-b7a7-0812dce80cf0\") " Jul 16 10:41:05.618658 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.618608 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mhbx\" (UniqueName: \"kubernetes.io/projected/e17b881e-42d4-4582-b9f4-6a35075a354b-kube-api-access-5mhbx\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:41:05.620400 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.620378 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5baea508-dd5d-4b31-b7a7-0812dce80cf0-kube-api-access-4t8tb" (OuterVolumeSpecName: "kube-api-access-4t8tb") pod "5baea508-dd5d-4b31-b7a7-0812dce80cf0" (UID: "5baea508-dd5d-4b31-b7a7-0812dce80cf0"). InnerVolumeSpecName "kube-api-access-4t8tb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:41:05.719971 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:05.719869 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t8tb\" (UniqueName: \"kubernetes.io/projected/5baea508-dd5d-4b31-b7a7-0812dce80cf0-kube-api-access-4t8tb\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:41:06.198470 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.198430 2581 generic.go:356] "Generic (PLEG): container finished" podID="5baea508-dd5d-4b31-b7a7-0812dce80cf0" containerID="fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045" exitCode=0 Jul 16 10:41:06.199131 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.198492 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-l9n8t" Jul 16 10:41:06.199131 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.198514 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-l9n8t" event={"ID":"5baea508-dd5d-4b31-b7a7-0812dce80cf0","Type":"ContainerDied","Data":"fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045"} Jul 16 10:41:06.199131 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.198554 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-l9n8t" event={"ID":"5baea508-dd5d-4b31-b7a7-0812dce80cf0","Type":"ContainerDied","Data":"34f7a4885c246910f1b53d686db3b7dd4be5762d0baa3fa0c2fd0808fbfb7513"} Jul 16 10:41:06.199131 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.198574 2581 scope.go:117] "RemoveContainer" containerID="fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045" Jul 16 10:41:06.199869 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.199846 2581 generic.go:356] "Generic (PLEG): container finished" podID="e17b881e-42d4-4582-b9f4-6a35075a354b" containerID="3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf" exitCode=0 Jul 16 10:41:06.199970 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.199877 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" event={"ID":"e17b881e-42d4-4582-b9f4-6a35075a354b","Type":"ContainerDied","Data":"3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf"} Jul 16 10:41:06.199970 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.199915 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" event={"ID":"e17b881e-42d4-4582-b9f4-6a35075a354b","Type":"ContainerDied","Data":"ba69fe4d20a418c45b2633b08bb47f5acd7f0ed8df0d125a45770c965b026fda"} Jul 16 10:41:06.199970 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.199913 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8476c4d5-w5bt8" Jul 16 10:41:06.207369 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.207350 2581 scope.go:117] "RemoveContainer" containerID="fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045" Jul 16 10:41:06.207610 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:41:06.207595 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045\": container with ID starting with fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045 not found: ID does not exist" containerID="fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045" Jul 16 10:41:06.207658 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.207618 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045"} err="failed to get container status \"fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045\": rpc error: code = NotFound desc = could not find container \"fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045\": container with ID starting with fbe487f820cf7c2e891341e8207c825301f8879a8f7bad1c8ef50a026edc0045 not found: ID does not exist" Jul 16 10:41:06.207658 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.207634 2581 scope.go:117] "RemoveContainer" containerID="3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf" Jul 16 10:41:06.215460 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.215438 2581 scope.go:117] "RemoveContainer" containerID="3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf" Jul 16 10:41:06.215741 ip-10-0-142-37 kubenswrapper[2581]: E0716 10:41:06.215708 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf\": container with ID starting with 3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf not found: ID does not exist" containerID="3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf" Jul 16 10:41:06.215798 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.215747 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf"} err="failed to get container status \"3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf\": rpc error: code = NotFound desc = could not find container \"3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf\": container with ID starting with 3e2f4e2ae8a6cb35255dbcd1cb5b8fda3d59d7e5f7bb59a43ddeaac1831098cf not found: ID does not exist" Jul 16 10:41:06.234516 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.234483 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l9n8t"] Jul 16 10:41:06.245510 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.245484 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l9n8t"] Jul 16 10:41:06.263655 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.263624 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d8476c4d5-w5bt8"] Jul 16 10:41:06.272557 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:06.272528 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d8476c4d5-w5bt8"] Jul 16 10:41:07.805435 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:07.805388 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5baea508-dd5d-4b31-b7a7-0812dce80cf0" path="/var/lib/kubelet/pods/5baea508-dd5d-4b31-b7a7-0812dce80cf0/volumes" Jul 16 10:41:07.805827 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:07.805744 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e17b881e-42d4-4582-b9f4-6a35075a354b" path="/var/lib/kubelet/pods/e17b881e-42d4-4582-b9f4-6a35075a354b/volumes" Jul 16 10:41:07.998603 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:07.998573 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55b9fdc6cc-x65rw"] Jul 16 10:41:07.998939 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:07.998927 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5baea508-dd5d-4b31-b7a7-0812dce80cf0" containerName="authorino" Jul 16 10:41:07.998939 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:07.998940 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5baea508-dd5d-4b31-b7a7-0812dce80cf0" containerName="authorino" Jul 16 10:41:07.999055 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:07.998950 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e17b881e-42d4-4582-b9f4-6a35075a354b" containerName="authorino" Jul 16 10:41:07.999055 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:07.998955 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e17b881e-42d4-4582-b9f4-6a35075a354b" containerName="authorino" Jul 16 10:41:07.999055 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:07.999019 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e17b881e-42d4-4582-b9f4-6a35075a354b" containerName="authorino" Jul 16 10:41:07.999055 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:07.999029 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5baea508-dd5d-4b31-b7a7-0812dce80cf0" containerName="authorino" Jul 16 10:41:08.003404 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.003388 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:08.006859 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.006827 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 10:41:08.006859 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.006838 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9j2bz\"" Jul 16 10:41:08.007016 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.006862 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 10:41:08.017464 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.017435 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55b9fdc6cc-x65rw"] Jul 16 10:41:08.141392 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.141346 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa3027c1-5c73-4af9-b9b1-8594e6b65876-webhook-cert\") pod \"maas-controller-55b9fdc6cc-x65rw\" (UID: \"aa3027c1-5c73-4af9-b9b1-8594e6b65876\") " pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:08.141599 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.141427 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs4ns\" (UniqueName: \"kubernetes.io/projected/aa3027c1-5c73-4af9-b9b1-8594e6b65876-kube-api-access-cs4ns\") pod \"maas-controller-55b9fdc6cc-x65rw\" (UID: \"aa3027c1-5c73-4af9-b9b1-8594e6b65876\") " pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:08.242910 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.242874 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs4ns\" (UniqueName: \"kubernetes.io/projected/aa3027c1-5c73-4af9-b9b1-8594e6b65876-kube-api-access-cs4ns\") pod \"maas-controller-55b9fdc6cc-x65rw\" (UID: \"aa3027c1-5c73-4af9-b9b1-8594e6b65876\") " pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:08.243096 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.242968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa3027c1-5c73-4af9-b9b1-8594e6b65876-webhook-cert\") pod \"maas-controller-55b9fdc6cc-x65rw\" (UID: \"aa3027c1-5c73-4af9-b9b1-8594e6b65876\") " pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:08.245424 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.245384 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa3027c1-5c73-4af9-b9b1-8594e6b65876-webhook-cert\") pod \"maas-controller-55b9fdc6cc-x65rw\" (UID: \"aa3027c1-5c73-4af9-b9b1-8594e6b65876\") " pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:08.252795 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.252775 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs4ns\" (UniqueName: \"kubernetes.io/projected/aa3027c1-5c73-4af9-b9b1-8594e6b65876-kube-api-access-cs4ns\") pod \"maas-controller-55b9fdc6cc-x65rw\" (UID: \"aa3027c1-5c73-4af9-b9b1-8594e6b65876\") " pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:08.313424 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.313374 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:08.450943 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:08.450919 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55b9fdc6cc-x65rw"] Jul 16 10:41:08.452141 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:41:08.452117 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa3027c1_5c73_4af9_b9b1_8594e6b65876.slice/crio-deda9bc34f6cf8c2aaa65eb34fa7cd3014e954c83a18d34e94ea23c6fe31577a WatchSource:0}: Error finding container deda9bc34f6cf8c2aaa65eb34fa7cd3014e954c83a18d34e94ea23c6fe31577a: Status 404 returned error can't find the container with id deda9bc34f6cf8c2aaa65eb34fa7cd3014e954c83a18d34e94ea23c6fe31577a Jul 16 10:41:09.219104 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:09.219063 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" event={"ID":"aa3027c1-5c73-4af9-b9b1-8594e6b65876","Type":"ContainerStarted","Data":"deda9bc34f6cf8c2aaa65eb34fa7cd3014e954c83a18d34e94ea23c6fe31577a"} Jul 16 10:41:11.230816 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:11.230714 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" event={"ID":"aa3027c1-5c73-4af9-b9b1-8594e6b65876","Type":"ContainerStarted","Data":"3b56405592087c236355c289e1d052c1f66549ceb665515d935b4019cef8184b"} Jul 16 10:41:11.230816 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:11.230785 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:11.254416 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:11.254349 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" podStartSLOduration=1.7759186649999998 podStartE2EDuration="4.254333536s" totalImagesPullingTime="2.478414871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:41:11.252036539 +0000 UTC m=+525.931301519" watchObservedRunningTime="2026-07-16 10:41:11.254333536 +0000 UTC m=+525.933598516" Jul 16 10:41:22.238941 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:22.238855 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:23.053535 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.053505 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55b9fdc6cc-x65rw"] Jul 16 10:41:23.053788 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.053674 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" podUID="aa3027c1-5c73-4af9-b9b1-8594e6b65876" containerName="manager" containerID="cri-o://3b56405592087c236355c289e1d052c1f66549ceb665515d935b4019cef8184b" gracePeriod=30 Jul 16 10:41:23.295540 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.295514 2581 generic.go:356] "Generic (PLEG): container finished" podID="aa3027c1-5c73-4af9-b9b1-8594e6b65876" containerID="3b56405592087c236355c289e1d052c1f66549ceb665515d935b4019cef8184b" exitCode=0 Jul 16 10:41:23.295889 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.295589 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" event={"ID":"aa3027c1-5c73-4af9-b9b1-8594e6b65876","Type":"ContainerDied","Data":"3b56405592087c236355c289e1d052c1f66549ceb665515d935b4019cef8184b"} Jul 16 10:41:23.295889 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.295654 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" event={"ID":"aa3027c1-5c73-4af9-b9b1-8594e6b65876","Type":"ContainerDied","Data":"deda9bc34f6cf8c2aaa65eb34fa7cd3014e954c83a18d34e94ea23c6fe31577a"} Jul 16 10:41:23.295889 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.295678 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="deda9bc34f6cf8c2aaa65eb34fa7cd3014e954c83a18d34e94ea23c6fe31577a" Jul 16 10:41:23.300983 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.300967 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:23.480228 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.480196 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa3027c1-5c73-4af9-b9b1-8594e6b65876-webhook-cert\") pod \"aa3027c1-5c73-4af9-b9b1-8594e6b65876\" (UID: \"aa3027c1-5c73-4af9-b9b1-8594e6b65876\") " Jul 16 10:41:23.480423 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.480302 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cs4ns\" (UniqueName: \"kubernetes.io/projected/aa3027c1-5c73-4af9-b9b1-8594e6b65876-kube-api-access-cs4ns\") pod \"aa3027c1-5c73-4af9-b9b1-8594e6b65876\" (UID: \"aa3027c1-5c73-4af9-b9b1-8594e6b65876\") " Jul 16 10:41:23.482494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.482458 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa3027c1-5c73-4af9-b9b1-8594e6b65876-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "aa3027c1-5c73-4af9-b9b1-8594e6b65876" (UID: "aa3027c1-5c73-4af9-b9b1-8594e6b65876"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 10:41:23.482602 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.482522 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa3027c1-5c73-4af9-b9b1-8594e6b65876-kube-api-access-cs4ns" (OuterVolumeSpecName: "kube-api-access-cs4ns") pod "aa3027c1-5c73-4af9-b9b1-8594e6b65876" (UID: "aa3027c1-5c73-4af9-b9b1-8594e6b65876"). InnerVolumeSpecName "kube-api-access-cs4ns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 10:41:23.580980 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.580948 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cs4ns\" (UniqueName: \"kubernetes.io/projected/aa3027c1-5c73-4af9-b9b1-8594e6b65876-kube-api-access-cs4ns\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:41:23.580980 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:23.580974 2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa3027c1-5c73-4af9-b9b1-8594e6b65876-webhook-cert\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 10:41:24.300793 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:24.300764 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b9fdc6cc-x65rw" Jul 16 10:41:24.320394 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:24.320367 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55b9fdc6cc-x65rw"] Jul 16 10:41:24.327433 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:24.327413 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55b9fdc6cc-x65rw"] Jul 16 10:41:25.805794 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:25.805761 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa3027c1-5c73-4af9-b9b1-8594e6b65876" path="/var/lib/kubelet/pods/aa3027c1-5c73-4af9-b9b1-8594e6b65876/volumes" Jul 16 10:41:43.656569 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.656537 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k"] Jul 16 10:41:43.657207 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.657000 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa3027c1-5c73-4af9-b9b1-8594e6b65876" containerName="manager" Jul 16 10:41:43.657207 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.657018 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa3027c1-5c73-4af9-b9b1-8594e6b65876" containerName="manager" Jul 16 10:41:43.657207 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.657111 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa3027c1-5c73-4af9-b9b1-8594e6b65876" containerName="manager" Jul 16 10:41:43.663656 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.663636 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:43.667091 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.667064 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 10:41:43.667202 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.667123 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 16 10:41:43.675226 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.675208 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 16 10:41:43.678741 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.678719 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k"] Jul 16 10:41:43.750315 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.750285 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ac9e004-d93d-41cc-82b3-819be495d5be-maas-api-tls\") pod \"maas-api-54d48cd479-tzh2k\" (UID: \"0ac9e004-d93d-41cc-82b3-819be495d5be\") " pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:43.750479 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.750326 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac9e004-d93d-41cc-82b3-819be495d5be-openshift-service-ca\") pod \"maas-api-54d48cd479-tzh2k\" (UID: \"0ac9e004-d93d-41cc-82b3-819be495d5be\") " pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:43.750479 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.750352 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69glh\" (UniqueName: \"kubernetes.io/projected/0ac9e004-d93d-41cc-82b3-819be495d5be-kube-api-access-69glh\") pod \"maas-api-54d48cd479-tzh2k\" (UID: \"0ac9e004-d93d-41cc-82b3-819be495d5be\") " pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:43.851257 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.851231 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ac9e004-d93d-41cc-82b3-819be495d5be-maas-api-tls\") pod \"maas-api-54d48cd479-tzh2k\" (UID: \"0ac9e004-d93d-41cc-82b3-819be495d5be\") " pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:43.851427 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.851267 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac9e004-d93d-41cc-82b3-819be495d5be-openshift-service-ca\") pod \"maas-api-54d48cd479-tzh2k\" (UID: \"0ac9e004-d93d-41cc-82b3-819be495d5be\") " pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:43.851427 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.851298 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69glh\" (UniqueName: \"kubernetes.io/projected/0ac9e004-d93d-41cc-82b3-819be495d5be-kube-api-access-69glh\") pod \"maas-api-54d48cd479-tzh2k\" (UID: \"0ac9e004-d93d-41cc-82b3-819be495d5be\") " pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:43.851852 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.851830 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac9e004-d93d-41cc-82b3-819be495d5be-openshift-service-ca\") pod \"maas-api-54d48cd479-tzh2k\" (UID: \"0ac9e004-d93d-41cc-82b3-819be495d5be\") " pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:43.853659 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.853638 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ac9e004-d93d-41cc-82b3-819be495d5be-maas-api-tls\") pod \"maas-api-54d48cd479-tzh2k\" (UID: \"0ac9e004-d93d-41cc-82b3-819be495d5be\") " pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:43.868494 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.868468 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69glh\" (UniqueName: \"kubernetes.io/projected/0ac9e004-d93d-41cc-82b3-819be495d5be-kube-api-access-69glh\") pod \"maas-api-54d48cd479-tzh2k\" (UID: \"0ac9e004-d93d-41cc-82b3-819be495d5be\") " pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:43.973929 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:43.973853 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:44.144646 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:41:44.144615 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac9e004_d93d_41cc_82b3_819be495d5be.slice/crio-5ba0c9e1e21d0efb1f8e1615ad7607c3e0aeb149672f81215de005c750acee53 WatchSource:0}: Error finding container 5ba0c9e1e21d0efb1f8e1615ad7607c3e0aeb149672f81215de005c750acee53: Status 404 returned error can't find the container with id 5ba0c9e1e21d0efb1f8e1615ad7607c3e0aeb149672f81215de005c750acee53 Jul 16 10:41:44.145044 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.145018 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k"] Jul 16 10:41:44.401680 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.401644 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" event={"ID":"0ac9e004-d93d-41cc-82b3-819be495d5be","Type":"ContainerStarted","Data":"5ba0c9e1e21d0efb1f8e1615ad7607c3e0aeb149672f81215de005c750acee53"} Jul 16 10:41:44.617355 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.617323 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-gvklw"] Jul 16 10:41:44.620538 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.620519 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" Jul 16 10:41:44.624419 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.624400 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8fv7p\"" Jul 16 10:41:44.624514 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.624433 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 16 10:41:44.638754 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.638732 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-gvklw"] Jul 16 10:41:44.658445 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.658379 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48rxp\" (UniqueName: \"kubernetes.io/projected/9d86f4dc-ea88-407b-8a60-6fc5c1ec6451-kube-api-access-48rxp\") pod \"payload-pre-processing-74584bb9f8-gvklw\" (UID: \"9d86f4dc-ea88-407b-8a60-6fc5c1ec6451\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" Jul 16 10:41:44.658445 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.658414 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d86f4dc-ea88-407b-8a60-6fc5c1ec6451-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-gvklw\" (UID: \"9d86f4dc-ea88-407b-8a60-6fc5c1ec6451\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" Jul 16 10:41:44.683407 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.683379 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-6rzvw"] Jul 16 10:41:44.686050 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.686029 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" Jul 16 10:41:44.689750 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.689732 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:41:44.702587 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.702564 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-6rzvw"] Jul 16 10:41:44.759721 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.759652 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/464f26a1-df36-4845-996c-ab2ec952e8f8-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-6rzvw\" (UID: \"464f26a1-df36-4845-996c-ab2ec952e8f8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" Jul 16 10:41:44.760964 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.759753 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48rxp\" (UniqueName: \"kubernetes.io/projected/9d86f4dc-ea88-407b-8a60-6fc5c1ec6451-kube-api-access-48rxp\") pod \"payload-pre-processing-74584bb9f8-gvklw\" (UID: \"9d86f4dc-ea88-407b-8a60-6fc5c1ec6451\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" Jul 16 10:41:44.760964 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.759790 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d86f4dc-ea88-407b-8a60-6fc5c1ec6451-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-gvklw\" (UID: \"9d86f4dc-ea88-407b-8a60-6fc5c1ec6451\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" Jul 16 10:41:44.760964 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.760387 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w269h\" (UniqueName: \"kubernetes.io/projected/464f26a1-df36-4845-996c-ab2ec952e8f8-kube-api-access-w269h\") pod \"payload-processing-f9b8d9b9b-6rzvw\" (UID: \"464f26a1-df36-4845-996c-ab2ec952e8f8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" Jul 16 10:41:44.760964 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.760639 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d86f4dc-ea88-407b-8a60-6fc5c1ec6451-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-gvklw\" (UID: \"9d86f4dc-ea88-407b-8a60-6fc5c1ec6451\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" Jul 16 10:41:44.772048 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.772019 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48rxp\" (UniqueName: \"kubernetes.io/projected/9d86f4dc-ea88-407b-8a60-6fc5c1ec6451-kube-api-access-48rxp\") pod \"payload-pre-processing-74584bb9f8-gvklw\" (UID: \"9d86f4dc-ea88-407b-8a60-6fc5c1ec6451\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" Jul 16 10:41:44.861220 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.861191 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w269h\" (UniqueName: \"kubernetes.io/projected/464f26a1-df36-4845-996c-ab2ec952e8f8-kube-api-access-w269h\") pod \"payload-processing-f9b8d9b9b-6rzvw\" (UID: \"464f26a1-df36-4845-996c-ab2ec952e8f8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" Jul 16 10:41:44.861385 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.861255 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/464f26a1-df36-4845-996c-ab2ec952e8f8-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-6rzvw\" (UID: \"464f26a1-df36-4845-996c-ab2ec952e8f8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" Jul 16 10:41:44.861905 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.861887 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/464f26a1-df36-4845-996c-ab2ec952e8f8-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-6rzvw\" (UID: \"464f26a1-df36-4845-996c-ab2ec952e8f8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" Jul 16 10:41:44.872474 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.872451 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w269h\" (UniqueName: \"kubernetes.io/projected/464f26a1-df36-4845-996c-ab2ec952e8f8-kube-api-access-w269h\") pod \"payload-processing-f9b8d9b9b-6rzvw\" (UID: \"464f26a1-df36-4845-996c-ab2ec952e8f8\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" Jul 16 10:41:44.930710 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.930460 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" Jul 16 10:41:44.999926 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:44.998659 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" Jul 16 10:41:45.115665 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:45.115630 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-gvklw"] Jul 16 10:41:45.118381 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:41:45.118329 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d86f4dc_ea88_407b_8a60_6fc5c1ec6451.slice/crio-6ef364fa94d826023dd8345b94ad671c850e04513fd6cc45731c2b98b1a9de03 WatchSource:0}: Error finding container 6ef364fa94d826023dd8345b94ad671c850e04513fd6cc45731c2b98b1a9de03: Status 404 returned error can't find the container with id 6ef364fa94d826023dd8345b94ad671c850e04513fd6cc45731c2b98b1a9de03 Jul 16 10:41:45.168622 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:45.168592 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-6rzvw"] Jul 16 10:41:45.169046 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:41:45.169017 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod464f26a1_df36_4845_996c_ab2ec952e8f8.slice/crio-42513a416199f34c8514881af9be7d7876459feaae7a8d103e95aad4c8000ff9 WatchSource:0}: Error finding container 42513a416199f34c8514881af9be7d7876459feaae7a8d103e95aad4c8000ff9: Status 404 returned error can't find the container with id 42513a416199f34c8514881af9be7d7876459feaae7a8d103e95aad4c8000ff9 Jul 16 10:41:45.409083 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:45.409051 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" event={"ID":"9d86f4dc-ea88-407b-8a60-6fc5c1ec6451","Type":"ContainerStarted","Data":"6ef364fa94d826023dd8345b94ad671c850e04513fd6cc45731c2b98b1a9de03"} Jul 16 10:41:45.410463 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:45.410434 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" event={"ID":"464f26a1-df36-4845-996c-ab2ec952e8f8","Type":"ContainerStarted","Data":"42513a416199f34c8514881af9be7d7876459feaae7a8d103e95aad4c8000ff9"} Jul 16 10:41:46.419978 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:46.419841 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" event={"ID":"0ac9e004-d93d-41cc-82b3-819be495d5be","Type":"ContainerStarted","Data":"a4b727770f614ae3a1b837417d3b777c8c8e66facc832280f4345ea3b332c0bb"} Jul 16 10:41:46.419978 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:46.419944 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:46.460893 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:46.457335 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" podStartSLOduration=1.8224996820000001 podStartE2EDuration="3.457315388s" totalImagesPullingTime="1.634815706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:41:46.455953684 +0000 UTC m=+561.135218662" watchObservedRunningTime="2026-07-16 10:41:46.457315388 +0000 UTC m=+561.136580367" Jul 16 10:41:47.426482 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:47.426443 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" event={"ID":"9d86f4dc-ea88-407b-8a60-6fc5c1ec6451","Type":"ContainerStarted","Data":"e7de7ee3aaf24cd11abb3e405856bee539686a6b41b76ed1437d4c49271847d6"} Jul 16 10:41:47.426954 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:47.426516 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" Jul 16 10:41:47.428059 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:47.428034 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" event={"ID":"464f26a1-df36-4845-996c-ab2ec952e8f8","Type":"ContainerStarted","Data":"eb130da539bc323750ef47fa80d806fb876a550fe07c1a67dcc6a4f8fa606355"} Jul 16 10:41:47.457780 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:47.457683 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" podStartSLOduration=1.71155256 podStartE2EDuration="3.457658541s" totalImagesPullingTime="1.746105981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:41:47.45621295 +0000 UTC m=+562.135477930" watchObservedRunningTime="2026-07-16 10:41:47.457658541 +0000 UTC m=+562.136923524" Jul 16 10:41:47.481431 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:47.481361 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" podStartSLOduration=1.788348419 podStartE2EDuration="3.481347121s" totalImagesPullingTime="1.692998702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:41:47.479209851 +0000 UTC m=+562.158474830" watchObservedRunningTime="2026-07-16 10:41:47.481347121 +0000 UTC m=+562.160612100" Jul 16 10:41:48.434135 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:48.434104 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" Jul 16 10:41:52.433814 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:52.433784 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54d48cd479-tzh2k" Jul 16 10:41:52.460585 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:52.460559 2581 generic.go:356] "Generic (PLEG): container finished" podID="e6bcaaaa-9f7e-4c89-8c94-c66486302286" containerID="402c1b5bc9a289cdc64053366c74bb3a11a3a44ee469383ecb73e80067ddb652" exitCode=0 Jul 16 10:41:52.460765 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:52.460636 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" event={"ID":"e6bcaaaa-9f7e-4c89-8c94-c66486302286","Type":"ContainerDied","Data":"402c1b5bc9a289cdc64053366c74bb3a11a3a44ee469383ecb73e80067ddb652"} Jul 16 10:41:52.461021 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:52.461007 2581 scope.go:117] "RemoveContainer" containerID="402c1b5bc9a289cdc64053366c74bb3a11a3a44ee469383ecb73e80067ddb652" Jul 16 10:41:52.462059 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:52.462034 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 10:41:52.462131 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:52.462097 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 10:41:52.462131 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:52.462127 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 10:41:53.468250 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:53.468220 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" event={"ID":"e6bcaaaa-9f7e-4c89-8c94-c66486302286","Type":"ContainerStarted","Data":"322b25b1f82e9a97745f33ca00985eff6130d9f7d267b1e25b2dcdca0f75730d"} Jul 16 10:41:53.649922 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:53.649881 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:41:53.650944 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:53.650912 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g82mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 16 10:41:53.651078 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:53.650967 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" podUID="e6bcaaaa-9f7e-4c89-8c94-c66486302286" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 16 10:41:54.649117 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:54.649079 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g82mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 16 10:41:54.649742 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:54.649707 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" podUID="e6bcaaaa-9f7e-4c89-8c94-c66486302286" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 16 10:41:55.649889 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:55.649844 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g82mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 16 10:41:55.650376 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:55.649930 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" podUID="e6bcaaaa-9f7e-4c89-8c94-c66486302286" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 16 10:41:56.649198 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:56.649157 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g82mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 16 10:41:56.649511 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:56.649477 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" podUID="e6bcaaaa-9f7e-4c89-8c94-c66486302286" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 16 10:41:56.969712 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:56.969598 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:41:57.653273 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:57.653244 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:41:57.654373 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:57.654350 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g82mq" Jul 16 10:41:58.436792 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:58.436761 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-gvklw" Jul 16 10:41:59.443418 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:41:59.443387 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-6rzvw" Jul 16 10:42:05.599905 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:05.599871 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:42:14.794180 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:14.794145 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:42:22.721659 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.721627 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq"] Jul 16 10:42:22.725220 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.725197 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:22.733132 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.733103 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6m4hr\"" Jul 16 10:42:22.733132 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.733106 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 10:42:22.733313 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.733145 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 10:42:22.733448 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.733432 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 16 10:42:22.739779 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.739755 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq"] Jul 16 10:42:22.903846 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.903814 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn7kl\" (UniqueName: \"kubernetes.io/projected/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-kube-api-access-pn7kl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:22.904019 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.903870 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:22.904019 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.903910 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:22.904019 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.903925 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:22.904019 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.903943 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:22.904175 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:22.904014 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.005097 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.005015 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn7kl\" (UniqueName: \"kubernetes.io/projected/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-kube-api-access-pn7kl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.005097 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.005070 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.005097 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.005091 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.005345 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.005107 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.005345 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.005126 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.005345 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.005151 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.005559 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.005535 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.005626 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.005565 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.005680 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.005627 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.007414 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.007398 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.007608 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.007587 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.014833 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.014798 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn7kl\" (UniqueName: \"kubernetes.io/projected/f1bb8e14-c388-4669-8d21-a2a0e772e6b1-kube-api-access-pn7kl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4gtxq\" (UID: \"f1bb8e14-c388-4669-8d21-a2a0e772e6b1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.035758 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.035730 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:23.169116 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.169092 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq"] Jul 16 10:42:23.170810 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:42:23.170786 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1bb8e14_c388_4669_8d21_a2a0e772e6b1.slice/crio-67cc5ece87d7f1f5f77811bcc1b1c4c6753ef8f37b1e85a45604234f3f9979a3 WatchSource:0}: Error finding container 67cc5ece87d7f1f5f77811bcc1b1c4c6753ef8f37b1e85a45604234f3f9979a3: Status 404 returned error can't find the container with id 67cc5ece87d7f1f5f77811bcc1b1c4c6753ef8f37b1e85a45604234f3f9979a3 Jul 16 10:42:23.628171 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:23.628130 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" event={"ID":"f1bb8e14-c388-4669-8d21-a2a0e772e6b1","Type":"ContainerStarted","Data":"67cc5ece87d7f1f5f77811bcc1b1c4c6753ef8f37b1e85a45604234f3f9979a3"} Jul 16 10:42:24.002733 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:24.002624 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:42:24.634861 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:24.634824 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" event={"ID":"f1bb8e14-c388-4669-8d21-a2a0e772e6b1","Type":"ContainerStarted","Data":"cee48bfb256bfc08e9f485e769c76b17d589576ac214cc9b1033f588ec3a7b7a"} Jul 16 10:42:24.658181 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:24.658130 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" podStartSLOduration=1.609089774 podStartE2EDuration="2.658111261s" totalImagesPullingTime="1.049021487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:42:24.655429722 +0000 UTC m=+599.334694702" watchObservedRunningTime="2026-07-16 10:42:24.658111261 +0000 UTC m=+599.337376240" Jul 16 10:42:26.587827 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:26.587794 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:42:26.588214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:26.587988 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:42:33.036762 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:33.036722 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:33.037169 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:33.036799 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:33.048962 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:33.048936 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:33.502551 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:33.502517 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:42:33.693008 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:33.692978 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4gtxq" Jul 16 10:42:38.910279 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:38.910244 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:42:48.209098 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:42:48.209013 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:44:00.287906 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.287873 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-564b4db568-jzcjj"] Jul 16 10:44:00.291276 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.291258 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-564b4db568-jzcjj" Jul 16 10:44:00.295476 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.295451 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fd6bn\"" Jul 16 10:44:00.295573 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.295488 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 16 10:44:00.306104 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.306081 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-564b4db568-jzcjj"] Jul 16 10:44:00.447081 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.447048 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/be35bc62-d858-4792-ac1f-7e904eddaf28-tls-cert\") pod \"authorino-564b4db568-jzcjj\" (UID: \"be35bc62-d858-4792-ac1f-7e904eddaf28\") " pod="kuadrant-system/authorino-564b4db568-jzcjj" Jul 16 10:44:00.447243 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.447105 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr9jh\" (UniqueName: \"kubernetes.io/projected/be35bc62-d858-4792-ac1f-7e904eddaf28-kube-api-access-kr9jh\") pod \"authorino-564b4db568-jzcjj\" (UID: \"be35bc62-d858-4792-ac1f-7e904eddaf28\") " pod="kuadrant-system/authorino-564b4db568-jzcjj" Jul 16 10:44:00.548571 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.548487 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr9jh\" (UniqueName: \"kubernetes.io/projected/be35bc62-d858-4792-ac1f-7e904eddaf28-kube-api-access-kr9jh\") pod \"authorino-564b4db568-jzcjj\" (UID: \"be35bc62-d858-4792-ac1f-7e904eddaf28\") " pod="kuadrant-system/authorino-564b4db568-jzcjj" Jul 16 10:44:00.548571 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.548563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/be35bc62-d858-4792-ac1f-7e904eddaf28-tls-cert\") pod \"authorino-564b4db568-jzcjj\" (UID: \"be35bc62-d858-4792-ac1f-7e904eddaf28\") " pod="kuadrant-system/authorino-564b4db568-jzcjj" Jul 16 10:44:00.551375 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.551345 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/be35bc62-d858-4792-ac1f-7e904eddaf28-tls-cert\") pod \"authorino-564b4db568-jzcjj\" (UID: \"be35bc62-d858-4792-ac1f-7e904eddaf28\") " pod="kuadrant-system/authorino-564b4db568-jzcjj" Jul 16 10:44:00.564510 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.564484 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr9jh\" (UniqueName: \"kubernetes.io/projected/be35bc62-d858-4792-ac1f-7e904eddaf28-kube-api-access-kr9jh\") pod \"authorino-564b4db568-jzcjj\" (UID: \"be35bc62-d858-4792-ac1f-7e904eddaf28\") " pod="kuadrant-system/authorino-564b4db568-jzcjj" Jul 16 10:44:00.600259 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.600227 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-564b4db568-jzcjj" Jul 16 10:44:00.743465 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.743431 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-564b4db568-jzcjj"] Jul 16 10:44:00.744617 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:44:00.744590 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe35bc62_d858_4792_ac1f_7e904eddaf28.slice/crio-b3a21ec774dc8439d272d08ce26450a076a725344656eb8346511c0c1bbe76a2 WatchSource:0}: Error finding container b3a21ec774dc8439d272d08ce26450a076a725344656eb8346511c0c1bbe76a2: Status 404 returned error can't find the container with id b3a21ec774dc8439d272d08ce26450a076a725344656eb8346511c0c1bbe76a2 Jul 16 10:44:00.745943 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:00.745924 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 10:44:01.141885 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:01.141848 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-564b4db568-jzcjj" event={"ID":"be35bc62-d858-4792-ac1f-7e904eddaf28","Type":"ContainerStarted","Data":"b3a21ec774dc8439d272d08ce26450a076a725344656eb8346511c0c1bbe76a2"} Jul 16 10:44:02.149252 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:02.149214 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-564b4db568-jzcjj" event={"ID":"be35bc62-d858-4792-ac1f-7e904eddaf28","Type":"ContainerStarted","Data":"cf2c1a8bc9f68d318674c9943cd814755cc75458898d4d73ebee3be2c307b873"} Jul 16 10:44:02.191098 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:02.191041 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-564b4db568-jzcjj" podStartSLOduration=1.785651739 podStartE2EDuration="2.191025997s" totalImagesPullingTime="405.374258ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:44:02.190144523 +0000 UTC m=+696.869409502" watchObservedRunningTime="2026-07-16 10:44:02.191025997 +0000 UTC m=+696.870290975" Jul 16 10:44:30.300373 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:30.300339 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:44:40.604351 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:40.604315 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:44:52.392037 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:44:52.392000 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:45:00.382159 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:00.382117 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:45:11.424260 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:11.424222 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:45:15.261479 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.261445 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c86f4fc58-7lwtw"] Jul 16 10:45:15.264929 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.264912 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" Jul 16 10:45:15.269741 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.269721 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9j2bz\"" Jul 16 10:45:15.269893 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.269794 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 10:45:15.270600 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.270579 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 10:45:15.281191 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.281172 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c86f4fc58-7lwtw"] Jul 16 10:45:15.284100 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.284077 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f34fe98-3d95-4f84-a011-087c61950369-webhook-cert\") pod \"maas-controller-6c86f4fc58-7lwtw\" (UID: \"8f34fe98-3d95-4f84-a011-087c61950369\") " pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" Jul 16 10:45:15.284178 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.284126 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsq2z\" (UniqueName: \"kubernetes.io/projected/8f34fe98-3d95-4f84-a011-087c61950369-kube-api-access-hsq2z\") pod \"maas-controller-6c86f4fc58-7lwtw\" (UID: \"8f34fe98-3d95-4f84-a011-087c61950369\") " pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" Jul 16 10:45:15.385252 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.385221 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsq2z\" (UniqueName: \"kubernetes.io/projected/8f34fe98-3d95-4f84-a011-087c61950369-kube-api-access-hsq2z\") pod \"maas-controller-6c86f4fc58-7lwtw\" (UID: \"8f34fe98-3d95-4f84-a011-087c61950369\") " pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" Jul 16 10:45:15.385428 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.385286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f34fe98-3d95-4f84-a011-087c61950369-webhook-cert\") pod \"maas-controller-6c86f4fc58-7lwtw\" (UID: \"8f34fe98-3d95-4f84-a011-087c61950369\") " pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" Jul 16 10:45:15.387599 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.387581 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f34fe98-3d95-4f84-a011-087c61950369-webhook-cert\") pod \"maas-controller-6c86f4fc58-7lwtw\" (UID: \"8f34fe98-3d95-4f84-a011-087c61950369\") " pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" Jul 16 10:45:15.396285 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.396262 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsq2z\" (UniqueName: \"kubernetes.io/projected/8f34fe98-3d95-4f84-a011-087c61950369-kube-api-access-hsq2z\") pod \"maas-controller-6c86f4fc58-7lwtw\" (UID: \"8f34fe98-3d95-4f84-a011-087c61950369\") " pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" Jul 16 10:45:15.575439 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.575404 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" Jul 16 10:45:15.718496 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:15.718468 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c86f4fc58-7lwtw"] Jul 16 10:45:15.719744 ip-10-0-142-37 kubenswrapper[2581]: W0716 10:45:15.719720 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f34fe98_3d95_4f84_a011_087c61950369.slice/crio-f9ac2a627a31f70069449c82b48b22bea162d50c383caface6d3b2fcfde3349c WatchSource:0}: Error finding container f9ac2a627a31f70069449c82b48b22bea162d50c383caface6d3b2fcfde3349c: Status 404 returned error can't find the container with id f9ac2a627a31f70069449c82b48b22bea162d50c383caface6d3b2fcfde3349c Jul 16 10:45:16.548767 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:16.548656 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" event={"ID":"8f34fe98-3d95-4f84-a011-087c61950369","Type":"ContainerStarted","Data":"dfa6a75a688ff25e6b4a932a459a92e3d37991c713a9b084ee08d940772d785a"} Jul 16 10:45:16.548767 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:16.548737 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" Jul 16 10:45:16.548767 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:16.548761 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" event={"ID":"8f34fe98-3d95-4f84-a011-087c61950369","Type":"ContainerStarted","Data":"f9ac2a627a31f70069449c82b48b22bea162d50c383caface6d3b2fcfde3349c"} Jul 16 10:45:16.572091 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:16.572047 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" podStartSLOduration=1.156355659 podStartE2EDuration="1.572032592s" totalImagesPullingTime="415.676933ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 10:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 10:45:16.571011723 +0000 UTC m=+771.250276702" watchObservedRunningTime="2026-07-16 10:45:16.572032592 +0000 UTC m=+771.251297561" Jul 16 10:45:27.557532 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:27.557501 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c86f4fc58-7lwtw" Jul 16 10:45:38.013416 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:38.013372 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:45:48.547594 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:45:48.547514 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:46:25.605997 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:46:25.605960 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:46:38.820511 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:46:38.820478 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:46:48.003001 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:46:48.002967 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:46:55.100862 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:46:55.100825 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:47:07.103311 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:47:07.103274 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:47:17.431748 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:47:17.431648 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:47:26.621808 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:47:26.621780 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:47:26.624920 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:47:26.624902 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:47:26.836063 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:47:26.836034 2581 scope.go:117] "RemoveContainer" containerID="3b56405592087c236355c289e1d052c1f66549ceb665515d935b4019cef8184b" Jul 16 10:47:29.611817 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:47:29.611783 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:47:39.693039 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:47:39.693004 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:48:01.424712 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:48:01.424663 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:48:10.504426 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:48:10.504389 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:48:34.900756 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:48:34.900720 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:48:47.314084 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:48:47.313974 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:49:04.702054 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:49:04.702022 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:49:14.605948 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:49:14.605913 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:49:24.098170 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:49:24.098132 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:49:33.605502 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:49:33.605467 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:49:43.792426 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:49:43.792388 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:51:31.349773 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:51:31.349730 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:51:41.897283 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:51:41.897241 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:51:52.420326 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:51:52.420289 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:52:03.573947 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:52:03.573904 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:52:16.200118 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:52:16.200085 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:52:26.644188 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:52:26.644149 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:52:26.668127 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:52:26.668094 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:52:26.673382 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:52:26.673356 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:52:34.321948 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:52:34.321907 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:52:46.025120 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:52:46.025076 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:52:56.420555 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:52:56.420519 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:53:05.531003 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:53:05.530967 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:53:16.719906 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:53:16.715995 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:53:41.220564 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:53:41.220531 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:53:49.008730 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:53:49.008668 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:53:59.424949 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:53:59.424912 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:54:08.808602 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:54:08.808564 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:54:19.741186 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:54:19.741144 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:54:33.508581 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:54:33.508546 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:54:45.993760 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:54:45.993717 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:54:55.001857 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:54:55.001772 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:55:09.216021 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:55:09.215981 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:55:17.793597 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:55:17.793560 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:55:31.010502 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:55:31.010466 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:55:39.710548 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:55:39.710509 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:55:51.595779 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:55:51.595739 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:56:02.403582 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:56:02.403545 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:56:13.494885 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:56:13.494846 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:56:22.400396 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:56:22.400313 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:56:35.411762 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:56:35.411732 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:56:46.914796 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:56:46.914753 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:56:58.101769 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:56:58.101730 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:57:05.706214 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:57:05.706179 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:57:18.904651 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:57:18.904615 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:57:26.709568 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:57:26.709539 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:57:26.716436 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:57:26.716414 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 10:57:28.291909 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:57:28.291874 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:57:40.695223 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:57:40.695186 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:57:55.306231 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:57:55.306197 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:58:05.201983 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:58:05.201944 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:58:12.395777 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:58:12.395743 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:58:22.213785 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:58:22.213751 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:58:31.697722 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:58:31.697670 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 10:58:41.629328 ip-10-0-142-37 kubenswrapper[2581]: I0716 10:58:41.629292 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:00:09.705531 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:00:09.705490 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:00:17.399448 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:00:17.399414 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:00:25.305534 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:00:25.305491 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:00:35.997552 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:00:35.997510 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:00:50.298165 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:00:50.298082 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:01:19.490070 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:01:19.490030 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:01:32.061173 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:01:32.061137 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:02:19.050875 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.050709 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4"] Jul 16 11:02:19.054926 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.054902 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.058170 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.058120 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d156b9-openshift-default-dockercfg-8w4fg\"" Jul 16 11:02:19.067896 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.067871 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4"] Jul 16 11:02:19.210802 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.210767 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-token\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.210802 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.210810 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-envoy\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.211053 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.210881 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-podinfo\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.211053 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.210916 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-credential-socket\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.211053 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.210988 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-certs\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.211053 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.211034 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba69a607-03f1-48d3-9ee5-847c13af58ae-istiod-ca-cert\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.211218 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.211085 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-data\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.211218 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.211120 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcfng\" (UniqueName: \"kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-kube-api-access-mcfng\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.211218 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.211152 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-socket\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.312499 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.312400 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-certs\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.312499 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.312452 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba69a607-03f1-48d3-9ee5-847c13af58ae-istiod-ca-cert\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.312777 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.312508 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-data\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.312777 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.312541 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcfng\" (UniqueName: \"kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-kube-api-access-mcfng\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.312777 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.312582 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-socket\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.312777 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.312613 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-token\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.312777 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.312636 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-envoy\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.312777 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.312716 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-podinfo\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.312777 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.312747 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-credential-socket\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.313096 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.312856 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-certs\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.313096 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.313048 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-socket\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.313235 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.313200 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-credential-socket\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.313327 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.313252 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-data\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.313327 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.313276 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba69a607-03f1-48d3-9ee5-847c13af58ae-istiod-ca-cert\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.315243 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.315220 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-envoy\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.315343 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.315327 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-podinfo\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.321814 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.321787 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-token\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.321814 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.321803 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcfng\" (UniqueName: \"kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-kube-api-access-mcfng\") pod \"e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.366731 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.366694 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:19.516957 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.516928 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4"] Jul 16 11:02:19.518130 ip-10-0-142-37 kubenswrapper[2581]: W0716 11:02:19.518103 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba69a607_03f1_48d3_9ee5_847c13af58ae.slice/crio-a346301df5219795228db333553b5e3e2fb48962ad4484a46e0628607cfbbecd WatchSource:0}: Error finding container a346301df5219795228db333553b5e3e2fb48962ad4484a46e0628607cfbbecd: Status 404 returned error can't find the container with id a346301df5219795228db333553b5e3e2fb48962ad4484a46e0628607cfbbecd Jul 16 11:02:19.520250 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.520216 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:02:19.520326 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.520284 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:02:19.520326 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:19.520314 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:02:20.198745 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:20.198712 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" event={"ID":"ba69a607-03f1-48d3-9ee5-847c13af58ae","Type":"ContainerStarted","Data":"31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5"} Jul 16 11:02:20.199136 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:20.198760 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" event={"ID":"ba69a607-03f1-48d3-9ee5-847c13af58ae","Type":"ContainerStarted","Data":"a346301df5219795228db333553b5e3e2fb48962ad4484a46e0628607cfbbecd"} Jul 16 11:02:20.221098 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:20.221048 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" podStartSLOduration=1.221028996 podStartE2EDuration="1.221028996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:02:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:02:20.21984962 +0000 UTC m=+1794.899114596" watchObservedRunningTime="2026-07-16 11:02:20.221028996 +0000 UTC m=+1794.900293975" Jul 16 11:02:20.367489 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:20.367457 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:20.372352 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:20.372328 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:20.385189 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:20.385164 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:02:21.204651 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:21.204617 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:21.205660 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:21.205643 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:26.752915 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:26.752885 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 11:02:26.761372 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:26.761350 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 11:02:32.715499 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:32.715464 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:02:34.289571 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:34.289478 2581 generic.go:356] "Generic (PLEG): container finished" podID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerID="31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5" exitCode=0 Jul 16 11:02:34.289571 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:34.289551 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" event={"ID":"ba69a607-03f1-48d3-9ee5-847c13af58ae","Type":"ContainerDied","Data":"31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5"} Jul 16 11:02:34.290155 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:34.289967 2581 scope.go:117] "RemoveContainer" containerID="31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5" Jul 16 11:02:34.290986 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:34.290956 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:02:34.291098 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:34.291024 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:02:34.291098 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:34.291058 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:02:35.297165 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:35.297126 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" event={"ID":"ba69a607-03f1-48d3-9ee5-847c13af58ae","Type":"ContainerStarted","Data":"917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021"} Jul 16 11:02:35.367426 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:35.367379 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:35.368528 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:35.368501 2581 patch_prober.go:28] interesting pod/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 16 11:02:35.368645 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:35.368552 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 16 11:02:36.205082 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:36.205026 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:36.368081 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:36.368047 2581 patch_prober.go:28] interesting pod/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 16 11:02:36.368567 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:36.368122 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 16 11:02:37.367661 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:37.367625 2581 patch_prober.go:28] interesting pod/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 16 11:02:37.367855 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:37.367716 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 16 11:02:38.367881 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:38.367845 2581 patch_prober.go:28] interesting pod/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 16 11:02:38.368269 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:38.367915 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 16 11:02:39.371595 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:39.371560 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:39.372634 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:39.372613 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:02:44.913192 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:44.913161 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:02:54.500333 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:02:54.500290 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:03:10.510038 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:03:10.509998 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:03:24.593518 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:03:24.593478 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:03:38.497635 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:03:38.497600 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:03:55.599674 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:03:55.599598 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:04:28.805141 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:28.805104 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:04:43.142215 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:43.142175 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:04:46.757919 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:46.757878 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4"] Jul 16 11:04:46.758427 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:46.758101 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerName="istio-proxy" containerID="cri-o://917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021" gracePeriod=30 Jul 16 11:04:52.009863 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.009833 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:04:52.055747 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.055640 2581 generic.go:356] "Generic (PLEG): container finished" podID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerID="917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021" exitCode=0 Jul 16 11:04:52.055747 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.055703 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" Jul 16 11:04:52.055747 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.055702 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" event={"ID":"ba69a607-03f1-48d3-9ee5-847c13af58ae","Type":"ContainerDied","Data":"917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021"} Jul 16 11:04:52.055985 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.055771 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4" event={"ID":"ba69a607-03f1-48d3-9ee5-847c13af58ae","Type":"ContainerDied","Data":"a346301df5219795228db333553b5e3e2fb48962ad4484a46e0628607cfbbecd"} Jul 16 11:04:52.055985 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.055803 2581 scope.go:117] "RemoveContainer" containerID="917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021" Jul 16 11:04:52.065132 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.065109 2581 scope.go:117] "RemoveContainer" containerID="31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5" Jul 16 11:04:52.073572 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.073548 2581 scope.go:117] "RemoveContainer" containerID="917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021" Jul 16 11:04:52.073926 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:04:52.073904 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021\": container with ID starting with 917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021 not found: ID does not exist" containerID="917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021" Jul 16 11:04:52.074035 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.073933 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021"} err="failed to get container status \"917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021\": rpc error: code = NotFound desc = could not find container \"917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021\": container with ID starting with 917bc7b00b09f40b68daa328be3da4cae21500a4db7f6d62a7b05614ca565021 not found: ID does not exist" Jul 16 11:04:52.074035 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.073952 2581 scope.go:117] "RemoveContainer" containerID="31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5" Jul 16 11:04:52.074240 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:04:52.074220 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5\": container with ID starting with 31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5 not found: ID does not exist" containerID="31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5" Jul 16 11:04:52.074285 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.074246 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5"} err="failed to get container status \"31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5\": rpc error: code = NotFound desc = could not find container \"31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5\": container with ID starting with 31d49a1776b006b5d8d1cf34f42c7049b47eb2c6ea5e5bef86076ff9827654a5 not found: ID does not exist" Jul 16 11:04:52.101726 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.101670 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-socket\") pod \"ba69a607-03f1-48d3-9ee5-847c13af58ae\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " Jul 16 11:04:52.101887 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.101734 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-data\") pod \"ba69a607-03f1-48d3-9ee5-847c13af58ae\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " Jul 16 11:04:52.101887 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.101764 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-credential-socket\") pod \"ba69a607-03f1-48d3-9ee5-847c13af58ae\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " Jul 16 11:04:52.101887 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.101791 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-podinfo\") pod \"ba69a607-03f1-48d3-9ee5-847c13af58ae\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " Jul 16 11:04:52.101887 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.101810 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba69a607-03f1-48d3-9ee5-847c13af58ae-istiod-ca-cert\") pod \"ba69a607-03f1-48d3-9ee5-847c13af58ae\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " Jul 16 11:04:52.101887 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.101840 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcfng\" (UniqueName: \"kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-kube-api-access-mcfng\") pod \"ba69a607-03f1-48d3-9ee5-847c13af58ae\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " Jul 16 11:04:52.102100 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.101938 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba69a607-03f1-48d3-9ee5-847c13af58ae" (UID: "ba69a607-03f1-48d3-9ee5-847c13af58ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:04:52.102100 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.101976 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-certs\") pod \"ba69a607-03f1-48d3-9ee5-847c13af58ae\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " Jul 16 11:04:52.102100 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.102037 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-envoy\") pod \"ba69a607-03f1-48d3-9ee5-847c13af58ae\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " Jul 16 11:04:52.102100 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.102096 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-token\") pod \"ba69a607-03f1-48d3-9ee5-847c13af58ae\" (UID: \"ba69a607-03f1-48d3-9ee5-847c13af58ae\") " Jul 16 11:04:52.102301 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.102166 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba69a607-03f1-48d3-9ee5-847c13af58ae" (UID: "ba69a607-03f1-48d3-9ee5-847c13af58ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:04:52.102359 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.102318 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba69a607-03f1-48d3-9ee5-847c13af58ae" (UID: "ba69a607-03f1-48d3-9ee5-847c13af58ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:04:52.102422 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.102396 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-credential-socket\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:04:52.102422 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.102413 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-certs\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:04:52.102522 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.102428 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-workload-socket\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:04:52.102646 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.102602 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba69a607-03f1-48d3-9ee5-847c13af58ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba69a607-03f1-48d3-9ee5-847c13af58ae" (UID: "ba69a607-03f1-48d3-9ee5-847c13af58ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 11:04:52.102821 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.102680 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba69a607-03f1-48d3-9ee5-847c13af58ae" (UID: "ba69a607-03f1-48d3-9ee5-847c13af58ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:04:52.104243 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.104220 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-kube-api-access-mcfng" (OuterVolumeSpecName: "kube-api-access-mcfng") pod "ba69a607-03f1-48d3-9ee5-847c13af58ae" (UID: "ba69a607-03f1-48d3-9ee5-847c13af58ae"). InnerVolumeSpecName "kube-api-access-mcfng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:04:52.104335 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.104267 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba69a607-03f1-48d3-9ee5-847c13af58ae" (UID: "ba69a607-03f1-48d3-9ee5-847c13af58ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 11:04:52.104335 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.104284 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba69a607-03f1-48d3-9ee5-847c13af58ae" (UID: "ba69a607-03f1-48d3-9ee5-847c13af58ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:04:52.104416 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.104324 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba69a607-03f1-48d3-9ee5-847c13af58ae" (UID: "ba69a607-03f1-48d3-9ee5-847c13af58ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:04:52.203835 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.203793 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-envoy\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:04:52.203835 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.203826 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-token\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:04:52.203835 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.203835 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-data\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:04:52.203835 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.203843 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba69a607-03f1-48d3-9ee5-847c13af58ae-istio-podinfo\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:04:52.204118 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.203852 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba69a607-03f1-48d3-9ee5-847c13af58ae-istiod-ca-cert\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:04:52.204118 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.203860 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcfng\" (UniqueName: \"kubernetes.io/projected/ba69a607-03f1-48d3-9ee5-847c13af58ae-kube-api-access-mcfng\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:04:52.381182 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.381147 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4"] Jul 16 11:04:52.386018 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.385990 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d156b9-openshift-default-8589d8c989-69cd4"] Jul 16 11:04:52.895045 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:52.895005 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:04:53.807054 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:53.807021 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" path="/var/lib/kubelet/pods/ba69a607-03f1-48d3-9ee5-847c13af58ae/volumes" Jul 16 11:04:55.870117 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.870084 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g"] Jul 16 11:04:55.870499 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.870486 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerName="istio-proxy" Jul 16 11:04:55.870499 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.870500 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerName="istio-proxy" Jul 16 11:04:55.870578 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.870508 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerName="istio-proxy" Jul 16 11:04:55.870578 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.870514 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerName="istio-proxy" Jul 16 11:04:55.870578 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.870567 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerName="istio-proxy" Jul 16 11:04:55.870578 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.870576 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba69a607-03f1-48d3-9ee5-847c13af58ae" containerName="istio-proxy" Jul 16 11:04:55.873518 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.873502 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:55.883270 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.883249 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-89289a-openshift-default-dockercfg-kdkfm\"" Jul 16 11:04:55.909195 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.909166 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g"] Jul 16 11:04:55.936635 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.936604 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-istio-token\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:55.936849 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.936654 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb74d056-a54d-4761-a9ef-d07f91780157-istiod-ca-cert\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:55.936849 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.936718 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-certs\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:55.936849 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.936742 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-envoy\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:55.936849 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.936767 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l7lf\" (UniqueName: \"kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-kube-api-access-4l7lf\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:55.936849 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.936793 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-credential-socket\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:55.937127 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.936916 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-data\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:55.937127 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.936977 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-socket\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:55.937127 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:55.937010 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb74d056-a54d-4761-a9ef-d07f91780157-istio-podinfo\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.038145 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.038101 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-istio-token\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.038353 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.038191 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb74d056-a54d-4761-a9ef-d07f91780157-istiod-ca-cert\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.038353 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.038223 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-certs\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.038353 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.038244 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-envoy\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.038353 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.038273 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l7lf\" (UniqueName: \"kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-kube-api-access-4l7lf\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.038621 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.038519 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-credential-socket\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.038621 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.038593 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-data\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.038751 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.038634 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-socket\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.038751 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.038664 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb74d056-a54d-4761-a9ef-d07f91780157-istio-podinfo\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.038855 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.038809 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-certs\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.039081 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.039059 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-data\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.039292 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.039273 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-credential-socket\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.039418 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.039397 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-socket\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.039721 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.039684 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb74d056-a54d-4761-a9ef-d07f91780157-istiod-ca-cert\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.041646 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.041613 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb74d056-a54d-4761-a9ef-d07f91780157-istio-podinfo\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.041767 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.041672 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-envoy\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.047818 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.047795 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l7lf\" (UniqueName: \"kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-kube-api-access-4l7lf\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.049572 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.049551 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-istio-token\") pod \"e2e-api-b-89289a-openshift-default-787978698c-2dq2g\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.184290 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.184186 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:56.351243 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.351209 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g"] Jul 16 11:04:56.354485 ip-10-0-142-37 kubenswrapper[2581]: W0716 11:04:56.354456 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb74d056_a54d_4761_a9ef_d07f91780157.slice/crio-7ceb6b9749ffe849c404c250b18e8e4de9f5eca74b7136ca7220e9ad8154a5f5 WatchSource:0}: Error finding container 7ceb6b9749ffe849c404c250b18e8e4de9f5eca74b7136ca7220e9ad8154a5f5: Status 404 returned error can't find the container with id 7ceb6b9749ffe849c404c250b18e8e4de9f5eca74b7136ca7220e9ad8154a5f5 Jul 16 11:04:56.356632 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.356594 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:04:56.356745 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.356666 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:04:56.356745 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:56.356715 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:04:57.089583 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:57.089549 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" event={"ID":"fb74d056-a54d-4761-a9ef-d07f91780157","Type":"ContainerStarted","Data":"dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c"} Jul 16 11:04:57.090000 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:57.089609 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" event={"ID":"fb74d056-a54d-4761-a9ef-d07f91780157","Type":"ContainerStarted","Data":"7ceb6b9749ffe849c404c250b18e8e4de9f5eca74b7136ca7220e9ad8154a5f5"} Jul 16 11:04:57.120210 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:57.120159 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" podStartSLOduration=2.120142869 podStartE2EDuration="2.120142869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:57.118883597 +0000 UTC m=+1951.798148580" watchObservedRunningTime="2026-07-16 11:04:57.120142869 +0000 UTC m=+1951.799407850" Jul 16 11:04:57.185280 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:57.185241 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:57.187060 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:57.187033 2581 patch_prober.go:28] interesting pod/e2e-api-b-89289a-openshift-default-787978698c-2dq2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 16 11:04:57.187191 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:57.187095 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 16 11:04:58.190323 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:58.190287 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:59.101663 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:59.101633 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:04:59.102645 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:04:59.102625 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:05:07.633171 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:07.633130 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:05:15.142880 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:15.142831 2581 patch_prober.go:28] interesting pod/e2e-api-b-89289a-openshift-default-787978698c-2dq2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 16 11:05:15.143310 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:15.142889 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 16 11:05:16.206575 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:16.206540 2581 generic.go:356] "Generic (PLEG): container finished" podID="fb74d056-a54d-4761-a9ef-d07f91780157" containerID="dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c" exitCode=0 Jul 16 11:05:16.207017 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:16.206598 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" event={"ID":"fb74d056-a54d-4761-a9ef-d07f91780157","Type":"ContainerDied","Data":"dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c"} Jul 16 11:05:16.207089 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:16.207074 2581 scope.go:117] "RemoveContainer" containerID="dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c" Jul 16 11:05:16.208134 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:16.208102 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:05:16.208207 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:16.208191 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:05:16.208253 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:16.208235 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:05:17.214722 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:17.214609 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" event={"ID":"fb74d056-a54d-4761-a9ef-d07f91780157","Type":"ContainerStarted","Data":"f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff"} Jul 16 11:05:18.185250 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:18.185210 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:05:18.185502 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:18.185481 2581 patch_prober.go:28] interesting pod/e2e-api-b-89289a-openshift-default-787978698c-2dq2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 16 11:05:18.185557 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:18.185538 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 16 11:05:19.185116 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:19.185071 2581 patch_prober.go:28] interesting pod/e2e-api-b-89289a-openshift-default-787978698c-2dq2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 16 11:05:19.185518 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:19.185151 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 16 11:05:20.185369 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:20.185325 2581 patch_prober.go:28] interesting pod/e2e-api-b-89289a-openshift-default-787978698c-2dq2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 16 11:05:20.185896 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:20.185399 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 16 11:05:21.189243 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:21.189206 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:05:21.189641 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:21.189584 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:05:21.190258 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:21.190235 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:05:22.601903 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:22.601872 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:05:36.894804 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:36.894767 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:05:40.322814 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:40.322778 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g"] Jul 16 11:05:40.323308 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:40.322978 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" containerID="cri-o://f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff" gracePeriod=30 Jul 16 11:05:45.572117 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.572092 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:05:45.607334 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607231 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb74d056-a54d-4761-a9ef-d07f91780157-istio-podinfo\") pod \"fb74d056-a54d-4761-a9ef-d07f91780157\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " Jul 16 11:05:45.607334 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607274 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-data\") pod \"fb74d056-a54d-4761-a9ef-d07f91780157\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " Jul 16 11:05:45.607334 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607299 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-certs\") pod \"fb74d056-a54d-4761-a9ef-d07f91780157\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " Jul 16 11:05:45.607592 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607349 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-socket\") pod \"fb74d056-a54d-4761-a9ef-d07f91780157\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " Jul 16 11:05:45.607592 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607377 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-credential-socket\") pod \"fb74d056-a54d-4761-a9ef-d07f91780157\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " Jul 16 11:05:45.607592 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607419 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-istio-token\") pod \"fb74d056-a54d-4761-a9ef-d07f91780157\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " Jul 16 11:05:45.607592 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607446 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb74d056-a54d-4761-a9ef-d07f91780157-istiod-ca-cert\") pod \"fb74d056-a54d-4761-a9ef-d07f91780157\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " Jul 16 11:05:45.607592 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607491 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-envoy\") pod \"fb74d056-a54d-4761-a9ef-d07f91780157\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " Jul 16 11:05:45.607592 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607544 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l7lf\" (UniqueName: \"kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-kube-api-access-4l7lf\") pod \"fb74d056-a54d-4761-a9ef-d07f91780157\" (UID: \"fb74d056-a54d-4761-a9ef-d07f91780157\") " Jul 16 11:05:45.607946 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607659 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fb74d056-a54d-4761-a9ef-d07f91780157" (UID: "fb74d056-a54d-4761-a9ef-d07f91780157"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:05:45.607946 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607658 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fb74d056-a54d-4761-a9ef-d07f91780157" (UID: "fb74d056-a54d-4761-a9ef-d07f91780157"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:05:45.607946 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.607678 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fb74d056-a54d-4761-a9ef-d07f91780157" (UID: "fb74d056-a54d-4761-a9ef-d07f91780157"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:05:45.608124 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.608018 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-certs\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:05:45.608124 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.608047 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-workload-socket\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:05:45.608124 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.608061 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-credential-socket\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:05:45.608124 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.608018 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb74d056-a54d-4761-a9ef-d07f91780157-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fb74d056-a54d-4761-a9ef-d07f91780157" (UID: "fb74d056-a54d-4761-a9ef-d07f91780157"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 11:05:45.608590 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.608314 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-data" (OuterVolumeSpecName: "istio-data") pod "fb74d056-a54d-4761-a9ef-d07f91780157" (UID: "fb74d056-a54d-4761-a9ef-d07f91780157"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:05:45.609720 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.609656 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-istio-token" (OuterVolumeSpecName: "istio-token") pod "fb74d056-a54d-4761-a9ef-d07f91780157" (UID: "fb74d056-a54d-4761-a9ef-d07f91780157"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:05:45.609994 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.609953 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-kube-api-access-4l7lf" (OuterVolumeSpecName: "kube-api-access-4l7lf") pod "fb74d056-a54d-4761-a9ef-d07f91780157" (UID: "fb74d056-a54d-4761-a9ef-d07f91780157"). InnerVolumeSpecName "kube-api-access-4l7lf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:05:45.610134 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.610122 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fb74d056-a54d-4761-a9ef-d07f91780157" (UID: "fb74d056-a54d-4761-a9ef-d07f91780157"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:05:45.610280 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.610255 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb74d056-a54d-4761-a9ef-d07f91780157-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fb74d056-a54d-4761-a9ef-d07f91780157" (UID: "fb74d056-a54d-4761-a9ef-d07f91780157"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 11:05:45.709456 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.709425 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-istio-token\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:05:45.709456 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.709451 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb74d056-a54d-4761-a9ef-d07f91780157-istiod-ca-cert\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:05:45.709456 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.709460 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-envoy\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:05:45.709714 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.709468 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l7lf\" (UniqueName: \"kubernetes.io/projected/fb74d056-a54d-4761-a9ef-d07f91780157-kube-api-access-4l7lf\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:05:45.709714 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.709477 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb74d056-a54d-4761-a9ef-d07f91780157-istio-podinfo\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:05:45.709714 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.709493 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb74d056-a54d-4761-a9ef-d07f91780157-istio-data\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:05:45.893054 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:45.892969 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:05:46.385235 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.385196 2581 generic.go:356] "Generic (PLEG): container finished" podID="fb74d056-a54d-4761-a9ef-d07f91780157" containerID="f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff" exitCode=0 Jul 16 11:05:46.385411 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.385249 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" Jul 16 11:05:46.385411 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.385269 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" event={"ID":"fb74d056-a54d-4761-a9ef-d07f91780157","Type":"ContainerDied","Data":"f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff"} Jul 16 11:05:46.385411 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.385321 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g" event={"ID":"fb74d056-a54d-4761-a9ef-d07f91780157","Type":"ContainerDied","Data":"7ceb6b9749ffe849c404c250b18e8e4de9f5eca74b7136ca7220e9ad8154a5f5"} Jul 16 11:05:46.385411 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.385353 2581 scope.go:117] "RemoveContainer" containerID="f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff" Jul 16 11:05:46.394585 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.394565 2581 scope.go:117] "RemoveContainer" containerID="dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c" Jul 16 11:05:46.402568 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.402546 2581 scope.go:117] "RemoveContainer" containerID="f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff" Jul 16 11:05:46.402802 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:05:46.402783 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff\": container with ID starting with f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff not found: ID does not exist" containerID="f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff" Jul 16 11:05:46.402866 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.402810 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff"} err="failed to get container status \"f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff\": rpc error: code = NotFound desc = could not find container \"f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff\": container with ID starting with f883831e6802d541dbb3f146eb032abebdf2547a630f7f49fa1ada55284fa0ff not found: ID does not exist" Jul 16 11:05:46.402866 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.402825 2581 scope.go:117] "RemoveContainer" containerID="dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c" Jul 16 11:05:46.403099 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:05:46.403081 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c\": container with ID starting with dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c not found: ID does not exist" containerID="dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c" Jul 16 11:05:46.403157 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.403104 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c"} err="failed to get container status \"dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c\": rpc error: code = NotFound desc = could not find container \"dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c\": container with ID starting with dde8e6e1840b22b8cb43187f08c526669b1d96b7b896513917020590f371691c not found: ID does not exist" Jul 16 11:05:46.410528 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.410498 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g"] Jul 16 11:05:46.417130 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:46.417110 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-89289a-openshift-default-787978698c-2dq2g"] Jul 16 11:05:47.807146 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:47.807111 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" path="/var/lib/kubelet/pods/fb74d056-a54d-4761-a9ef-d07f91780157/volumes" Jul 16 11:05:48.981279 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:48.981239 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g"] Jul 16 11:05:48.981774 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:48.981664 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" Jul 16 11:05:48.981774 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:48.981681 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" Jul 16 11:05:48.981774 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:48.981715 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" Jul 16 11:05:48.981774 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:48.981721 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" Jul 16 11:05:48.982011 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:48.981789 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" Jul 16 11:05:48.982011 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:48.981802 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb74d056-a54d-4761-a9ef-d07f91780157" containerName="istio-proxy" Jul 16 11:05:48.985190 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:48.985172 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:48.988061 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:48.988038 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-307109-openshift-default-dockercfg-jwh6b\"" Jul 16 11:05:48.997671 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:48.997649 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g"] Jul 16 11:05:49.034654 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.034616 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-socket\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.034654 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.034655 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-certs\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.034965 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.034683 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-podinfo\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.034965 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.034824 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-token\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.034965 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.034872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxjkj\" (UniqueName: \"kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-kube-api-access-mxjkj\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.034965 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.034941 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-credential-socket\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.035210 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.034973 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-envoy\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.035210 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.035026 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-data\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.035210 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.035089 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istiod-ca-cert\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.136146 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.136100 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-credential-socket\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.136359 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.136160 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-envoy\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.136359 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.136202 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-data\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.136493 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.136357 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istiod-ca-cert\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.136493 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.136410 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-socket\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.136493 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.136434 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-certs\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.136493 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.136467 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-podinfo\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.136740 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.136541 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-token\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.136740 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.136575 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjkj\" (UniqueName: \"kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-kube-api-access-mxjkj\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.136876 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.136857 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-credential-socket\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.137193 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.136873 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-data\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.137193 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.137142 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-socket\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.137193 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.137086 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-certs\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.137433 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.137408 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istiod-ca-cert\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.138993 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.138972 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-envoy\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.139449 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.139426 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-podinfo\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.145583 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.145554 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-token\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.145995 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.145975 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxjkj\" (UniqueName: \"kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-kube-api-access-mxjkj\") pod \"e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.297542 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.297445 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:49.464664 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.464632 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g"] Jul 16 11:05:49.467413 ip-10-0-142-37 kubenswrapper[2581]: W0716 11:05:49.467385 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d1bfb91_4497_4eab_b0d8_a5ac02ca59a5.slice/crio-e4e95380058d54de5d1366aa2f883e6f89adfa350bfd0831ab27425e0680fe05 WatchSource:0}: Error finding container e4e95380058d54de5d1366aa2f883e6f89adfa350bfd0831ab27425e0680fe05: Status 404 returned error can't find the container with id e4e95380058d54de5d1366aa2f883e6f89adfa350bfd0831ab27425e0680fe05 Jul 16 11:05:49.469607 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.469572 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:05:49.469679 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.469645 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:05:49.469737 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:49.469722 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:05:50.413256 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:50.413215 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" event={"ID":"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5","Type":"ContainerStarted","Data":"1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45"} Jul 16 11:05:50.413664 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:50.413289 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" event={"ID":"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5","Type":"ContainerStarted","Data":"e4e95380058d54de5d1366aa2f883e6f89adfa350bfd0831ab27425e0680fe05"} Jul 16 11:05:50.441447 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:50.441384 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" podStartSLOduration=2.441361883 podStartE2EDuration="2.441361883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:05:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:05:50.438519019 +0000 UTC m=+2005.117784001" watchObservedRunningTime="2026-07-16 11:05:50.441361883 +0000 UTC m=+2005.120626864" Jul 16 11:05:51.298564 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:51.298517 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:51.303837 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:51.303808 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:51.426718 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:51.420718 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:51.429165 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:51.428963 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:05:59.603367 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:05:59.603330 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:06:07.460898 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:07.460849 2581 patch_prober.go:28] interesting pod/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 16 11:06:07.461317 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:07.460929 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 16 11:06:08.516505 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:08.516396 2581 generic.go:356] "Generic (PLEG): container finished" podID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerID="1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45" exitCode=0 Jul 16 11:06:08.516505 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:08.516469 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" event={"ID":"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5","Type":"ContainerDied","Data":"1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45"} Jul 16 11:06:08.517042 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:08.516903 2581 scope.go:117] "RemoveContainer" containerID="1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45" Jul 16 11:06:08.517907 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:08.517881 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:06:08.517965 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:08.517946 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:06:08.518001 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:08.517977 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 11:06:09.523728 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:09.523673 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" event={"ID":"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5","Type":"ContainerStarted","Data":"1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61"} Jul 16 11:06:10.298052 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:10.298018 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:06:10.298233 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:10.298218 2581 patch_prober.go:28] interesting pod/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 16 11:06:10.298286 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:10.298257 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 16 11:06:11.298039 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:11.298005 2581 patch_prober.go:28] interesting pod/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 16 11:06:11.298442 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:11.298057 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 16 11:06:12.298202 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:12.298167 2581 patch_prober.go:28] interesting pod/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 16 11:06:12.298591 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:12.298233 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 16 11:06:12.304124 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:12.304095 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:06:13.297938 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:13.297902 2581 patch_prober.go:28] interesting pod/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 16 11:06:13.298127 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:13.297962 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 16 11:06:14.301795 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:14.301766 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:06:14.302263 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:14.302144 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:06:14.302901 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:14.302879 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:06:42.192259 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:42.192221 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:06:56.442165 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:06:56.442071 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:07:14.625791 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:07:14.625760 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:07:26.789433 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:07:26.789402 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 11:07:26.799429 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:07:26.799402 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 11:07:33.047550 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:07:33.047513 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:07:51.410031 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:07:51.409992 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:08:08.099009 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:08:08.098969 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:08:25.704071 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:08:25.703979 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:08:42.140553 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:08:42.140511 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:09:00.005655 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:09:00.001561 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:09:28.807003 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:09:28.806966 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:10:15.966589 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:10:15.966540 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:10:33.538497 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:10:33.538462 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:10:51.628570 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:10:51.628529 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:11:06.794415 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:11:06.794378 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:11:32.083930 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:11:32.083843 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:11:56.795367 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:11:56.795329 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:12:11.489857 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:11.489815 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:12:19.366022 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:19.365988 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7557-140051044121136"] Jul 16 11:12:19.369820 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:19.369800 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051044121136" Jul 16 11:12:19.374281 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:19.374253 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mqpb2\"" Jul 16 11:12:19.387975 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:19.387940 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051044121136"] Jul 16 11:12:19.512244 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:19.512205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cbkk\" (UniqueName: \"kubernetes.io/projected/ba85dc38-339c-45fc-8ffc-2a1e4313d300-kube-api-access-6cbkk\") pod \"test-curl-7557-140051044121136\" (UID: \"ba85dc38-339c-45fc-8ffc-2a1e4313d300\") " pod="opendatahub/test-curl-7557-140051044121136" Jul 16 11:12:19.613239 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:19.613206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cbkk\" (UniqueName: \"kubernetes.io/projected/ba85dc38-339c-45fc-8ffc-2a1e4313d300-kube-api-access-6cbkk\") pod \"test-curl-7557-140051044121136\" (UID: \"ba85dc38-339c-45fc-8ffc-2a1e4313d300\") " pod="opendatahub/test-curl-7557-140051044121136" Jul 16 11:12:19.621779 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:19.621684 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cbkk\" (UniqueName: \"kubernetes.io/projected/ba85dc38-339c-45fc-8ffc-2a1e4313d300-kube-api-access-6cbkk\") pod \"test-curl-7557-140051044121136\" (UID: \"ba85dc38-339c-45fc-8ffc-2a1e4313d300\") " pod="opendatahub/test-curl-7557-140051044121136" Jul 16 11:12:19.685367 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:19.685337 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051044121136" Jul 16 11:12:19.823222 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:19.823193 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051044121136"] Jul 16 11:12:19.824400 ip-10-0-142-37 kubenswrapper[2581]: W0716 11:12:19.824374 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba85dc38_339c_45fc_8ffc_2a1e4313d300.slice/crio-c605e58fbc2067a1c517f02e3ef2398cf69bc9965bda5dc62ecb4645dc986a58 WatchSource:0}: Error finding container c605e58fbc2067a1c517f02e3ef2398cf69bc9965bda5dc62ecb4645dc986a58: Status 404 returned error can't find the container with id c605e58fbc2067a1c517f02e3ef2398cf69bc9965bda5dc62ecb4645dc986a58 Jul 16 11:12:19.825653 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:19.825636 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 11:12:20.581991 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:20.581960 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051044121136" event={"ID":"ba85dc38-339c-45fc-8ffc-2a1e4313d300","Type":"ContainerStarted","Data":"098ab01164d23478149ab6f667cf059630c49c90ff6958d994233a5261f06f75"} Jul 16 11:12:20.582332 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:20.582008 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051044121136" event={"ID":"ba85dc38-339c-45fc-8ffc-2a1e4313d300","Type":"ContainerStarted","Data":"c605e58fbc2067a1c517f02e3ef2398cf69bc9965bda5dc62ecb4645dc986a58"} Jul 16 11:12:20.599181 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:20.599137 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7557-140051044121136" podStartSLOduration=0.920072903 podStartE2EDuration="1.599120829s" totalImagesPullingTime="679.047926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:12:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:12:20.597923928 +0000 UTC m=+2395.277188909" watchObservedRunningTime="2026-07-16 11:12:20.599120829 +0000 UTC m=+2395.278385808" Jul 16 11:12:20.721547 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:20.721500 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7557-140051044121136_ba85dc38-339c-45fc-8ffc-2a1e4313d300/test-curl-7557-140051044121136/0.log" Jul 16 11:12:21.588138 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:21.588104 2581 generic.go:356] "Generic (PLEG): container finished" podID="ba85dc38-339c-45fc-8ffc-2a1e4313d300" containerID="098ab01164d23478149ab6f667cf059630c49c90ff6958d994233a5261f06f75" exitCode=0 Jul 16 11:12:21.588535 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:21.588199 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051044121136" event={"ID":"ba85dc38-339c-45fc-8ffc-2a1e4313d300","Type":"ContainerDied","Data":"098ab01164d23478149ab6f667cf059630c49c90ff6958d994233a5261f06f75"} Jul 16 11:12:22.735066 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.735040 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051044121136" Jul 16 11:12:22.781218 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.781178 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7557-140051044121136"] Jul 16 11:12:22.788427 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.788398 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7557-140051044121136"] Jul 16 11:12:22.843364 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.843338 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cbkk\" (UniqueName: \"kubernetes.io/projected/ba85dc38-339c-45fc-8ffc-2a1e4313d300-kube-api-access-6cbkk\") pod \"ba85dc38-339c-45fc-8ffc-2a1e4313d300\" (UID: \"ba85dc38-339c-45fc-8ffc-2a1e4313d300\") " Jul 16 11:12:22.845427 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.845357 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba85dc38-339c-45fc-8ffc-2a1e4313d300-kube-api-access-6cbkk" (OuterVolumeSpecName: "kube-api-access-6cbkk") pod "ba85dc38-339c-45fc-8ffc-2a1e4313d300" (UID: "ba85dc38-339c-45fc-8ffc-2a1e4313d300"). InnerVolumeSpecName "kube-api-access-6cbkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:12:22.919758 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.919724 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7557-140051044187312"] Jul 16 11:12:22.920133 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.920120 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba85dc38-339c-45fc-8ffc-2a1e4313d300" containerName="test-curl-7557-140051044121136" Jul 16 11:12:22.920180 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.920135 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba85dc38-339c-45fc-8ffc-2a1e4313d300" containerName="test-curl-7557-140051044121136" Jul 16 11:12:22.920256 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.920196 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba85dc38-339c-45fc-8ffc-2a1e4313d300" containerName="test-curl-7557-140051044121136" Jul 16 11:12:22.923111 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.923096 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051044187312" Jul 16 11:12:22.931380 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.931358 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051044187312"] Jul 16 11:12:22.944296 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:22.944267 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cbkk\" (UniqueName: \"kubernetes.io/projected/ba85dc38-339c-45fc-8ffc-2a1e4313d300-kube-api-access-6cbkk\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:12:23.045629 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:23.045589 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wqpd\" (UniqueName: \"kubernetes.io/projected/4c6c7a17-0084-45a6-9056-1b15e32bf9c2-kube-api-access-8wqpd\") pod \"test-curl-7557-140051044187312\" (UID: \"4c6c7a17-0084-45a6-9056-1b15e32bf9c2\") " pod="opendatahub/test-curl-7557-140051044187312" Jul 16 11:12:23.146260 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:23.146167 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wqpd\" (UniqueName: \"kubernetes.io/projected/4c6c7a17-0084-45a6-9056-1b15e32bf9c2-kube-api-access-8wqpd\") pod \"test-curl-7557-140051044187312\" (UID: \"4c6c7a17-0084-45a6-9056-1b15e32bf9c2\") " pod="opendatahub/test-curl-7557-140051044187312" Jul 16 11:12:23.156707 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:23.156658 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wqpd\" (UniqueName: \"kubernetes.io/projected/4c6c7a17-0084-45a6-9056-1b15e32bf9c2-kube-api-access-8wqpd\") pod \"test-curl-7557-140051044187312\" (UID: \"4c6c7a17-0084-45a6-9056-1b15e32bf9c2\") " pod="opendatahub/test-curl-7557-140051044187312" Jul 16 11:12:23.233528 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:23.233496 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051044187312" Jul 16 11:12:23.369629 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:23.369599 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051044187312"] Jul 16 11:12:23.372423 ip-10-0-142-37 kubenswrapper[2581]: W0716 11:12:23.372396 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c6c7a17_0084_45a6_9056_1b15e32bf9c2.slice/crio-bb056df9c7ef58230dd0227c6be9f2ddc24c3ca2069214afd4d44685380f7783 WatchSource:0}: Error finding container bb056df9c7ef58230dd0227c6be9f2ddc24c3ca2069214afd4d44685380f7783: Status 404 returned error can't find the container with id bb056df9c7ef58230dd0227c6be9f2ddc24c3ca2069214afd4d44685380f7783 Jul 16 11:12:23.606509 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:23.606473 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051044121136" Jul 16 11:12:23.606727 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:23.606519 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c605e58fbc2067a1c517f02e3ef2398cf69bc9965bda5dc62ecb4645dc986a58" Jul 16 11:12:23.607817 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:23.607788 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051044187312" event={"ID":"4c6c7a17-0084-45a6-9056-1b15e32bf9c2","Type":"ContainerStarted","Data":"bb056df9c7ef58230dd0227c6be9f2ddc24c3ca2069214afd4d44685380f7783"} Jul 16 11:12:23.617574 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:12:23.617547 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7557-140051044121136\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="ba85dc38-339c-45fc-8ffc-2a1e4313d300" pod="opendatahub/test-curl-7557-140051044121136" Jul 16 11:12:23.806665 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:23.806629 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba85dc38-339c-45fc-8ffc-2a1e4313d300" path="/var/lib/kubelet/pods/ba85dc38-339c-45fc-8ffc-2a1e4313d300/volumes" Jul 16 11:12:24.614665 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:24.614630 2581 generic.go:356] "Generic (PLEG): container finished" podID="4c6c7a17-0084-45a6-9056-1b15e32bf9c2" containerID="a9dfdecfc311a8d2deb31eb5bb386565b85fc6a85c0e4784c3a6daceceea1c8c" exitCode=0 Jul 16 11:12:24.614975 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:24.614721 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051044187312" event={"ID":"4c6c7a17-0084-45a6-9056-1b15e32bf9c2","Type":"ContainerDied","Data":"a9dfdecfc311a8d2deb31eb5bb386565b85fc6a85c0e4784c3a6daceceea1c8c"} Jul 16 11:12:25.756116 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.756093 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051044187312" Jul 16 11:12:25.779545 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.779509 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7557-140051044187312_4c6c7a17-0084-45a6-9056-1b15e32bf9c2/test-curl-7557-140051044187312/0.log" Jul 16 11:12:25.811264 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.811238 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7557-140051044187312"] Jul 16 11:12:25.814029 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.814005 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7557-140051044187312"] Jul 16 11:12:25.871965 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.871880 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wqpd\" (UniqueName: \"kubernetes.io/projected/4c6c7a17-0084-45a6-9056-1b15e32bf9c2-kube-api-access-8wqpd\") pod \"4c6c7a17-0084-45a6-9056-1b15e32bf9c2\" (UID: \"4c6c7a17-0084-45a6-9056-1b15e32bf9c2\") " Jul 16 11:12:25.874091 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.874058 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c6c7a17-0084-45a6-9056-1b15e32bf9c2-kube-api-access-8wqpd" (OuterVolumeSpecName: "kube-api-access-8wqpd") pod "4c6c7a17-0084-45a6-9056-1b15e32bf9c2" (UID: "4c6c7a17-0084-45a6-9056-1b15e32bf9c2"). InnerVolumeSpecName "kube-api-access-8wqpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:12:25.973362 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.973336 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wqpd\" (UniqueName: \"kubernetes.io/projected/4c6c7a17-0084-45a6-9056-1b15e32bf9c2-kube-api-access-8wqpd\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:12:25.978395 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.978361 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7557-140051043532064"] Jul 16 11:12:25.978793 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.978778 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c6c7a17-0084-45a6-9056-1b15e32bf9c2" containerName="test-curl-7557-140051044187312" Jul 16 11:12:25.978843 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.978795 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c6c7a17-0084-45a6-9056-1b15e32bf9c2" containerName="test-curl-7557-140051044187312" Jul 16 11:12:25.978878 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.978852 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c6c7a17-0084-45a6-9056-1b15e32bf9c2" containerName="test-curl-7557-140051044187312" Jul 16 11:12:25.981763 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.981749 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:25.989798 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:25.989771 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051043532064"] Jul 16 11:12:26.074780 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:26.074746 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-972fc\" (UniqueName: \"kubernetes.io/projected/94547d93-916b-4e24-bf4d-4a5546c97b56-kube-api-access-972fc\") pod \"test-curl-7557-140051043532064\" (UID: \"94547d93-916b-4e24-bf4d-4a5546c97b56\") " pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:26.176288 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:26.176199 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-972fc\" (UniqueName: \"kubernetes.io/projected/94547d93-916b-4e24-bf4d-4a5546c97b56-kube-api-access-972fc\") pod \"test-curl-7557-140051043532064\" (UID: \"94547d93-916b-4e24-bf4d-4a5546c97b56\") " pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:26.185252 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:26.185217 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-972fc\" (UniqueName: \"kubernetes.io/projected/94547d93-916b-4e24-bf4d-4a5546c97b56-kube-api-access-972fc\") pod \"test-curl-7557-140051043532064\" (UID: \"94547d93-916b-4e24-bf4d-4a5546c97b56\") " pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:26.291059 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:26.291014 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:26.425069 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:26.425040 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051043532064"] Jul 16 11:12:26.426632 ip-10-0-142-37 kubenswrapper[2581]: W0716 11:12:26.426578 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94547d93_916b_4e24_bf4d_4a5546c97b56.slice/crio-0d0f33c9bd3ff9c10d0ef7e7a5168b08c57eb8515febac21b5d024dd411f2fc8 WatchSource:0}: Error finding container 0d0f33c9bd3ff9c10d0ef7e7a5168b08c57eb8515febac21b5d024dd411f2fc8: Status 404 returned error can't find the container with id 0d0f33c9bd3ff9c10d0ef7e7a5168b08c57eb8515febac21b5d024dd411f2fc8 Jul 16 11:12:26.627004 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:26.626979 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb056df9c7ef58230dd0227c6be9f2ddc24c3ca2069214afd4d44685380f7783" Jul 16 11:12:26.627004 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:26.626984 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051044187312" Jul 16 11:12:26.628234 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:26.628206 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051043532064" event={"ID":"94547d93-916b-4e24-bf4d-4a5546c97b56","Type":"ContainerStarted","Data":"0d0f33c9bd3ff9c10d0ef7e7a5168b08c57eb8515febac21b5d024dd411f2fc8"} Jul 16 11:12:26.637806 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:12:26.637780 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7557-140051044187312\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="4c6c7a17-0084-45a6-9056-1b15e32bf9c2" pod="opendatahub/test-curl-7557-140051044187312" Jul 16 11:12:26.834537 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:26.834513 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 11:12:26.847202 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:26.847186 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4n6zq_f72393fd-77d3-4b0d-85c6-310f88907af1/ovn-acl-logging/0.log" Jul 16 11:12:27.635160 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:27.635128 2581 generic.go:356] "Generic (PLEG): container finished" podID="94547d93-916b-4e24-bf4d-4a5546c97b56" containerID="58f2049aaa84c530dbd93300aba70c1fb16a02a9ba0b99d48900b0d133968360" exitCode=0 Jul 16 11:12:27.635335 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:27.635196 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051043532064" event={"ID":"94547d93-916b-4e24-bf4d-4a5546c97b56","Type":"ContainerDied","Data":"58f2049aaa84c530dbd93300aba70c1fb16a02a9ba0b99d48900b0d133968360"} Jul 16 11:12:27.663628 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:12:27.663593 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7557-140051044187312\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="4c6c7a17-0084-45a6-9056-1b15e32bf9c2" pod="opendatahub/test-curl-7557-140051044187312" Jul 16 11:12:27.806664 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:27.806630 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c6c7a17-0084-45a6-9056-1b15e32bf9c2" path="/var/lib/kubelet/pods/4c6c7a17-0084-45a6-9056-1b15e32bf9c2/volumes" Jul 16 11:12:28.617757 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.617720 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:12:28.775130 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.775104 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:28.803622 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.803595 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7557-140051043532064_94547d93-916b-4e24-bf4d-4a5546c97b56/test-curl-7557-140051043532064/0.log" Jul 16 11:12:28.834408 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.834385 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7557-140051043532064"] Jul 16 11:12:28.840087 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.840066 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7557-140051043532064"] Jul 16 11:12:28.899797 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.899726 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-972fc\" (UniqueName: \"kubernetes.io/projected/94547d93-916b-4e24-bf4d-4a5546c97b56-kube-api-access-972fc\") pod \"94547d93-916b-4e24-bf4d-4a5546c97b56\" (UID: \"94547d93-916b-4e24-bf4d-4a5546c97b56\") " Jul 16 11:12:28.901802 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.901771 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94547d93-916b-4e24-bf4d-4a5546c97b56-kube-api-access-972fc" (OuterVolumeSpecName: "kube-api-access-972fc") pod "94547d93-916b-4e24-bf4d-4a5546c97b56" (UID: "94547d93-916b-4e24-bf4d-4a5546c97b56"). InnerVolumeSpecName "kube-api-access-972fc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:12:28.961365 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.961335 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7557-140051043532064"] Jul 16 11:12:28.961791 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.961774 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94547d93-916b-4e24-bf4d-4a5546c97b56" containerName="test-curl-7557-140051043532064" Jul 16 11:12:28.961842 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.961793 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="94547d93-916b-4e24-bf4d-4a5546c97b56" containerName="test-curl-7557-140051043532064" Jul 16 11:12:28.961882 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.961862 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="94547d93-916b-4e24-bf4d-4a5546c97b56" containerName="test-curl-7557-140051043532064" Jul 16 11:12:28.964977 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.964962 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:28.971727 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:28.971677 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051043532064"] Jul 16 11:12:29.000650 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:29.000618 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-972fc\" (UniqueName: \"kubernetes.io/projected/94547d93-916b-4e24-bf4d-4a5546c97b56-kube-api-access-972fc\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:12:29.101793 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:29.101758 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbs4h\" (UniqueName: \"kubernetes.io/projected/c99db3a6-a579-40ab-a5e2-10a9aa96939c-kube-api-access-lbs4h\") pod \"test-curl-7557-140051043532064\" (UID: \"c99db3a6-a579-40ab-a5e2-10a9aa96939c\") " pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:29.203001 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:29.202901 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbs4h\" (UniqueName: \"kubernetes.io/projected/c99db3a6-a579-40ab-a5e2-10a9aa96939c-kube-api-access-lbs4h\") pod \"test-curl-7557-140051043532064\" (UID: \"c99db3a6-a579-40ab-a5e2-10a9aa96939c\") " pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:29.213276 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:29.213246 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbs4h\" (UniqueName: \"kubernetes.io/projected/c99db3a6-a579-40ab-a5e2-10a9aa96939c-kube-api-access-lbs4h\") pod \"test-curl-7557-140051043532064\" (UID: \"c99db3a6-a579-40ab-a5e2-10a9aa96939c\") " pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:29.274835 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:29.274799 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:29.412221 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:29.412197 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051043532064"] Jul 16 11:12:29.414036 ip-10-0-142-37 kubenswrapper[2581]: W0716 11:12:29.414011 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc99db3a6_a579_40ab_a5e2_10a9aa96939c.slice/crio-f56de2075e394cb9b14a487d59a68f2d62f8de2ebea604862c1e544c9a075ae0 WatchSource:0}: Error finding container f56de2075e394cb9b14a487d59a68f2d62f8de2ebea604862c1e544c9a075ae0: Status 404 returned error can't find the container with id f56de2075e394cb9b14a487d59a68f2d62f8de2ebea604862c1e544c9a075ae0 Jul 16 11:12:29.647816 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:29.647785 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:29.648227 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:29.647837 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d0f33c9bd3ff9c10d0ef7e7a5168b08c57eb8515febac21b5d024dd411f2fc8" Jul 16 11:12:29.649118 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:29.649085 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051043532064" event={"ID":"c99db3a6-a579-40ab-a5e2-10a9aa96939c","Type":"ContainerStarted","Data":"f56de2075e394cb9b14a487d59a68f2d62f8de2ebea604862c1e544c9a075ae0"} Jul 16 11:12:29.659495 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:29.659471 2581 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7557-140051043532064" oldPodUID="94547d93-916b-4e24-bf4d-4a5546c97b56" podUID="c99db3a6-a579-40ab-a5e2-10a9aa96939c" Jul 16 11:12:29.806217 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:29.806188 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94547d93-916b-4e24-bf4d-4a5546c97b56" path="/var/lib/kubelet/pods/94547d93-916b-4e24-bf4d-4a5546c97b56/volumes" Jul 16 11:12:30.655888 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:30.655851 2581 generic.go:356] "Generic (PLEG): container finished" podID="c99db3a6-a579-40ab-a5e2-10a9aa96939c" containerID="91a2ee7ba1af70b86f36dd183888175f2bae2c6125f6b782632475e137893c08" exitCode=0 Jul 16 11:12:30.656270 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:30.655936 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051043532064" event={"ID":"c99db3a6-a579-40ab-a5e2-10a9aa96939c","Type":"ContainerDied","Data":"91a2ee7ba1af70b86f36dd183888175f2bae2c6125f6b782632475e137893c08"} Jul 16 11:12:31.796549 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.796524 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:31.823316 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.823292 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7557-140051043532064_c99db3a6-a579-40ab-a5e2-10a9aa96939c/test-curl-7557-140051043532064/0.log" Jul 16 11:12:31.853370 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.853339 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7557-140051043532064"] Jul 16 11:12:31.858205 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.858175 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7557-140051043532064"] Jul 16 11:12:31.926020 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.925944 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbs4h\" (UniqueName: \"kubernetes.io/projected/c99db3a6-a579-40ab-a5e2-10a9aa96939c-kube-api-access-lbs4h\") pod \"c99db3a6-a579-40ab-a5e2-10a9aa96939c\" (UID: \"c99db3a6-a579-40ab-a5e2-10a9aa96939c\") " Jul 16 11:12:31.927992 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.927955 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c99db3a6-a579-40ab-a5e2-10a9aa96939c-kube-api-access-lbs4h" (OuterVolumeSpecName: "kube-api-access-lbs4h") pod "c99db3a6-a579-40ab-a5e2-10a9aa96939c" (UID: "c99db3a6-a579-40ab-a5e2-10a9aa96939c"). InnerVolumeSpecName "kube-api-access-lbs4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:12:31.987491 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.987455 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7557-140051043545424"] Jul 16 11:12:31.987906 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.987892 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c99db3a6-a579-40ab-a5e2-10a9aa96939c" containerName="test-curl-7557-140051043532064" Jul 16 11:12:31.987956 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.987908 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c99db3a6-a579-40ab-a5e2-10a9aa96939c" containerName="test-curl-7557-140051043532064" Jul 16 11:12:31.987992 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.987973 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c99db3a6-a579-40ab-a5e2-10a9aa96939c" containerName="test-curl-7557-140051043532064" Jul 16 11:12:31.990929 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.990913 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:31.996604 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:31.996573 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051043545424"] Jul 16 11:12:32.027521 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:32.027488 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbs4h\" (UniqueName: \"kubernetes.io/projected/c99db3a6-a579-40ab-a5e2-10a9aa96939c-kube-api-access-lbs4h\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:12:32.128480 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:32.128441 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwbjg\" (UniqueName: \"kubernetes.io/projected/cc7f7c9c-f74c-453a-af7e-8922b5ad112b-kube-api-access-mwbjg\") pod \"test-curl-7557-140051043545424\" (UID: \"cc7f7c9c-f74c-453a-af7e-8922b5ad112b\") " pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:32.229030 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:32.228945 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwbjg\" (UniqueName: \"kubernetes.io/projected/cc7f7c9c-f74c-453a-af7e-8922b5ad112b-kube-api-access-mwbjg\") pod \"test-curl-7557-140051043545424\" (UID: \"cc7f7c9c-f74c-453a-af7e-8922b5ad112b\") " pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:32.239188 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:32.239164 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwbjg\" (UniqueName: \"kubernetes.io/projected/cc7f7c9c-f74c-453a-af7e-8922b5ad112b-kube-api-access-mwbjg\") pod \"test-curl-7557-140051043545424\" (UID: \"cc7f7c9c-f74c-453a-af7e-8922b5ad112b\") " pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:32.301198 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:32.301174 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:32.435136 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:32.435110 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051043545424"] Jul 16 11:12:32.437280 ip-10-0-142-37 kubenswrapper[2581]: W0716 11:12:32.437257 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc7f7c9c_f74c_453a_af7e_8922b5ad112b.slice/crio-70e7dd777b27bfad034fb45cefeffa2a3a7173b5a0a1ae97bab78330bb628b7d WatchSource:0}: Error finding container 70e7dd777b27bfad034fb45cefeffa2a3a7173b5a0a1ae97bab78330bb628b7d: Status 404 returned error can't find the container with id 70e7dd777b27bfad034fb45cefeffa2a3a7173b5a0a1ae97bab78330bb628b7d Jul 16 11:12:32.668738 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:32.668678 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051043545424" event={"ID":"cc7f7c9c-f74c-453a-af7e-8922b5ad112b","Type":"ContainerStarted","Data":"70e7dd777b27bfad034fb45cefeffa2a3a7173b5a0a1ae97bab78330bb628b7d"} Jul 16 11:12:32.669912 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:32.669894 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f56de2075e394cb9b14a487d59a68f2d62f8de2ebea604862c1e544c9a075ae0" Jul 16 11:12:32.669912 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:32.669910 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:32.680605 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:12:32.680575 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7557-140051043532064\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="c99db3a6-a579-40ab-a5e2-10a9aa96939c" pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:33.677378 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:33.677345 2581 generic.go:356] "Generic (PLEG): container finished" podID="cc7f7c9c-f74c-453a-af7e-8922b5ad112b" containerID="fa7d3398213e132006746f82cbdf4ec31218a2a97b824cb8847b7781aa330ff5" exitCode=0 Jul 16 11:12:33.677789 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:33.677397 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051043545424" event={"ID":"cc7f7c9c-f74c-453a-af7e-8922b5ad112b","Type":"ContainerDied","Data":"fa7d3398213e132006746f82cbdf4ec31218a2a97b824cb8847b7781aa330ff5"} Jul 16 11:12:33.681528 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:12:33.681497 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7557-140051043532064\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="c99db3a6-a579-40ab-a5e2-10a9aa96939c" pod="opendatahub/test-curl-7557-140051043532064" Jul 16 11:12:33.805893 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:33.805862 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c99db3a6-a579-40ab-a5e2-10a9aa96939c" path="/var/lib/kubelet/pods/c99db3a6-a579-40ab-a5e2-10a9aa96939c/volumes" Jul 16 11:12:34.822475 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:34.822450 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:34.844889 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:34.844852 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7557-140051043545424_cc7f7c9c-f74c-453a-af7e-8922b5ad112b/test-curl-7557-140051043545424/0.log" Jul 16 11:12:34.872304 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:34.872270 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7557-140051043545424"] Jul 16 11:12:34.875895 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:34.875873 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7557-140051043545424"] Jul 16 11:12:34.955024 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:34.954936 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwbjg\" (UniqueName: \"kubernetes.io/projected/cc7f7c9c-f74c-453a-af7e-8922b5ad112b-kube-api-access-mwbjg\") pod \"cc7f7c9c-f74c-453a-af7e-8922b5ad112b\" (UID: \"cc7f7c9c-f74c-453a-af7e-8922b5ad112b\") " Jul 16 11:12:34.957003 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:34.956984 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc7f7c9c-f74c-453a-af7e-8922b5ad112b-kube-api-access-mwbjg" (OuterVolumeSpecName: "kube-api-access-mwbjg") pod "cc7f7c9c-f74c-453a-af7e-8922b5ad112b" (UID: "cc7f7c9c-f74c-453a-af7e-8922b5ad112b"). InnerVolumeSpecName "kube-api-access-mwbjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:12:34.999200 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:34.999170 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7557-140051043545424"] Jul 16 11:12:34.999560 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:34.999547 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc7f7c9c-f74c-453a-af7e-8922b5ad112b" containerName="test-curl-7557-140051043545424" Jul 16 11:12:34.999602 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:34.999561 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc7f7c9c-f74c-453a-af7e-8922b5ad112b" containerName="test-curl-7557-140051043545424" Jul 16 11:12:34.999637 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:34.999626 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc7f7c9c-f74c-453a-af7e-8922b5ad112b" containerName="test-curl-7557-140051043545424" Jul 16 11:12:35.002603 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.002588 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:35.008182 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.008159 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051043545424"] Jul 16 11:12:35.056182 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.056144 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts844\" (UniqueName: \"kubernetes.io/projected/44229895-c23c-4161-af67-3efb5ca90fa7-kube-api-access-ts844\") pod \"test-curl-7557-140051043545424\" (UID: \"44229895-c23c-4161-af67-3efb5ca90fa7\") " pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:35.056337 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.056240 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwbjg\" (UniqueName: \"kubernetes.io/projected/cc7f7c9c-f74c-453a-af7e-8922b5ad112b-kube-api-access-mwbjg\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:12:35.157576 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.157538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts844\" (UniqueName: \"kubernetes.io/projected/44229895-c23c-4161-af67-3efb5ca90fa7-kube-api-access-ts844\") pod \"test-curl-7557-140051043545424\" (UID: \"44229895-c23c-4161-af67-3efb5ca90fa7\") " pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:35.166853 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.166817 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts844\" (UniqueName: \"kubernetes.io/projected/44229895-c23c-4161-af67-3efb5ca90fa7-kube-api-access-ts844\") pod \"test-curl-7557-140051043545424\" (UID: \"44229895-c23c-4161-af67-3efb5ca90fa7\") " pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:35.312761 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.312673 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:35.447652 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.447624 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7557-140051043545424"] Jul 16 11:12:35.448468 ip-10-0-142-37 kubenswrapper[2581]: W0716 11:12:35.448443 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44229895_c23c_4161_af67_3efb5ca90fa7.slice/crio-f1c2ebedd55f5b6f48a5cb637d6078cece2bd981fcfdca35f7dc9383c34ee7ed WatchSource:0}: Error finding container f1c2ebedd55f5b6f48a5cb637d6078cece2bd981fcfdca35f7dc9383c34ee7ed: Status 404 returned error can't find the container with id f1c2ebedd55f5b6f48a5cb637d6078cece2bd981fcfdca35f7dc9383c34ee7ed Jul 16 11:12:35.690899 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.690870 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:35.690899 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.690894 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70e7dd777b27bfad034fb45cefeffa2a3a7173b5a0a1ae97bab78330bb628b7d" Jul 16 11:12:35.692143 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.692115 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051043545424" event={"ID":"44229895-c23c-4161-af67-3efb5ca90fa7","Type":"ContainerStarted","Data":"f1c2ebedd55f5b6f48a5cb637d6078cece2bd981fcfdca35f7dc9383c34ee7ed"} Jul 16 11:12:35.702737 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.702684 2581 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7557-140051043545424" oldPodUID="cc7f7c9c-f74c-453a-af7e-8922b5ad112b" podUID="44229895-c23c-4161-af67-3efb5ca90fa7" Jul 16 11:12:35.807239 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:35.807182 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc7f7c9c-f74c-453a-af7e-8922b5ad112b" path="/var/lib/kubelet/pods/cc7f7c9c-f74c-453a-af7e-8922b5ad112b/volumes" Jul 16 11:12:36.700424 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:36.700388 2581 generic.go:356] "Generic (PLEG): container finished" podID="44229895-c23c-4161-af67-3efb5ca90fa7" containerID="1ce09cd8a5c5b50989173ea4c775e807e1599f95d45a6d94bfea7a12ff121448" exitCode=0 Jul 16 11:12:36.700872 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:36.700471 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7557-140051043545424" event={"ID":"44229895-c23c-4161-af67-3efb5ca90fa7","Type":"ContainerDied","Data":"1ce09cd8a5c5b50989173ea4c775e807e1599f95d45a6d94bfea7a12ff121448"} Jul 16 11:12:37.857249 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:37.857221 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:37.885772 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:37.885741 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts844\" (UniqueName: \"kubernetes.io/projected/44229895-c23c-4161-af67-3efb5ca90fa7-kube-api-access-ts844\") pod \"44229895-c23c-4161-af67-3efb5ca90fa7\" (UID: \"44229895-c23c-4161-af67-3efb5ca90fa7\") " Jul 16 11:12:37.888084 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:37.888051 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44229895-c23c-4161-af67-3efb5ca90fa7-kube-api-access-ts844" (OuterVolumeSpecName: "kube-api-access-ts844") pod "44229895-c23c-4161-af67-3efb5ca90fa7" (UID: "44229895-c23c-4161-af67-3efb5ca90fa7"). InnerVolumeSpecName "kube-api-access-ts844". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:12:37.895903 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:37.895875 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7557-140051043545424_44229895-c23c-4161-af67-3efb5ca90fa7/test-curl-7557-140051043545424/0.log" Jul 16 11:12:37.936712 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:37.936655 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7557-140051043545424"] Jul 16 11:12:37.957246 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:37.957172 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7557-140051043545424"] Jul 16 11:12:37.987006 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:37.986976 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts844\" (UniqueName: \"kubernetes.io/projected/44229895-c23c-4161-af67-3efb5ca90fa7-kube-api-access-ts844\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:12:38.714224 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:38.714196 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:38.714384 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:38.714246 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1c2ebedd55f5b6f48a5cb637d6078cece2bd981fcfdca35f7dc9383c34ee7ed" Jul 16 11:12:38.724286 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:12:38.724255 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7557-140051043545424\" is forbidden: User \"system:node:ip-10-0-142-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-37.ec2.internal' and this object" podUID="44229895-c23c-4161-af67-3efb5ca90fa7" pod="opendatahub/test-curl-7557-140051043545424" Jul 16 11:12:39.807020 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:39.806985 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44229895-c23c-4161-af67-3efb5ca90fa7" path="/var/lib/kubelet/pods/44229895-c23c-4161-af67-3efb5ca90fa7/volumes" Jul 16 11:12:47.507655 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:12:47.507564 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:13:01.916115 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:01.916065 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrfw2"] Jul 16 11:13:06.344902 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:06.344867 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g"] Jul 16 11:13:06.345336 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:06.345091 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" containerID="cri-o://1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61" gracePeriod=30 Jul 16 11:13:07.014554 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:07.014519 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-564b4db568-jzcjj_be35bc62-d858-4792-ac1f-7e904eddaf28/authorino/0.log" Jul 16 11:13:11.386233 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:13:11.386201 2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d1bfb91_4497_4eab_b0d8_a5ac02ca59a5.slice/crio-1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d1bfb91_4497_4eab_b0d8_a5ac02ca59a5.slice/crio-conmon-1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61.scope\": RecentStats: unable to find data in memory cache]" Jul 16 11:13:11.605251 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.605223 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:13:11.787777 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.787656 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-token\") pod \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " Jul 16 11:13:11.787777 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.787725 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-credential-socket\") pod \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " Jul 16 11:13:11.787777 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.787749 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-envoy\") pod \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " Jul 16 11:13:11.787777 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.787777 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-certs\") pod \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " Jul 16 11:13:11.788108 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.787815 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-socket\") pod \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " Jul 16 11:13:11.788108 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.787837 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxjkj\" (UniqueName: \"kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-kube-api-access-mxjkj\") pod \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " Jul 16 11:13:11.788108 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.787858 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-data\") pod \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " Jul 16 11:13:11.788108 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.787904 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-podinfo\") pod \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " Jul 16 11:13:11.788108 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.787925 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istiod-ca-cert\") pod \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\" (UID: \"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5\") " Jul 16 11:13:11.788108 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.788067 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" (UID: "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:13:11.788413 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.788094 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" (UID: "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:13:11.788413 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.788308 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-credential-socket\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:13:11.788413 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.788327 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-certs\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:13:11.788413 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.788401 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" (UID: "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 11:13:11.788627 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.788467 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" (UID: "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:13:11.788898 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.788875 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-data" (OuterVolumeSpecName: "istio-data") pod "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" (UID: "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:13:11.789941 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.789919 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-token" (OuterVolumeSpecName: "istio-token") pod "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" (UID: "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:13:11.790167 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.790152 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" (UID: "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 11:13:11.790385 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.790362 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" (UID: "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 11:13:11.790484 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.790393 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-kube-api-access-mxjkj" (OuterVolumeSpecName: "kube-api-access-mxjkj") pod "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" (UID: "5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5"). InnerVolumeSpecName "kube-api-access-mxjkj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 11:13:11.889422 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.889387 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-token\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:13:11.889422 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.889416 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-envoy\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:13:11.889422 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.889427 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-workload-socket\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:13:11.889669 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.889440 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxjkj\" (UniqueName: \"kubernetes.io/projected/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-kube-api-access-mxjkj\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:13:11.889669 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.889453 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-data\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:13:11.889669 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.889463 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istio-podinfo\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:13:11.889669 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.889480 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5-istiod-ca-cert\") on node \"ip-10-0-142-37.ec2.internal\" DevicePath \"\"" Jul 16 11:13:11.899837 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.899806 2581 generic.go:356] "Generic (PLEG): container finished" podID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerID="1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61" exitCode=0 Jul 16 11:13:11.899982 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.899861 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" Jul 16 11:13:11.899982 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.899894 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" event={"ID":"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5","Type":"ContainerDied","Data":"1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61"} Jul 16 11:13:11.899982 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.899945 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g" event={"ID":"5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5","Type":"ContainerDied","Data":"e4e95380058d54de5d1366aa2f883e6f89adfa350bfd0831ab27425e0680fe05"} Jul 16 11:13:11.899982 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.899968 2581 scope.go:117] "RemoveContainer" containerID="1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61" Jul 16 11:13:11.909298 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.909279 2581 scope.go:117] "RemoveContainer" containerID="1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45" Jul 16 11:13:11.917208 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.917193 2581 scope.go:117] "RemoveContainer" containerID="1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61" Jul 16 11:13:11.917437 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:13:11.917414 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61\": container with ID starting with 1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61 not found: ID does not exist" containerID="1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61" Jul 16 11:13:11.917531 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.917444 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61"} err="failed to get container status \"1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61\": rpc error: code = NotFound desc = could not find container \"1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61\": container with ID starting with 1eca00a09f3ee048e0d54c8d9e36ae726f6cad5e7c5913800a9dc130fd530b61 not found: ID does not exist" Jul 16 11:13:11.917531 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.917460 2581 scope.go:117] "RemoveContainer" containerID="1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45" Jul 16 11:13:11.917713 ip-10-0-142-37 kubenswrapper[2581]: E0716 11:13:11.917681 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45\": container with ID starting with 1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45 not found: ID does not exist" containerID="1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45" Jul 16 11:13:11.917751 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.917719 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45"} err="failed to get container status \"1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45\": rpc error: code = NotFound desc = could not find container \"1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45\": container with ID starting with 1c2730f8f2d91646b98fdbbc6660633e249f63427ca59e71eca623e2be4b8b45 not found: ID does not exist" Jul 16 11:13:11.923921 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.923900 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g"] Jul 16 11:13:11.928882 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:11.928864 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-307109-openshift-default-846d8d99cf-cx88g"] Jul 16 11:13:12.634804 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:12.634766 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7rw8c_536d56ea-4060-4215-b36b-9b7e2373466c/manager/0.log" Jul 16 11:13:12.759711 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:12.759660 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tkmkm_8632f28b-52ee-4b02-9106-1bc9363c4387/manager/0.log" Jul 16 11:13:12.894644 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:12.894567 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c86f4fc58-7lwtw_8f34fe98-3d95-4f84-a011-087c61950369/manager/0.log" Jul 16 11:13:13.020463 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:13.020435 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5q7wq_99c1a236-aca2-4120-9a1d-473b2a4ef53e/server/0.log" Jul 16 11:13:13.151969 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:13.151889 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-d9mmp_08dc4380-9e09-4e3a-ab75-dd064a191ebe/manager/0.log" Jul 16 11:13:13.407600 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:13.407520 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-tntpt_eb815613-68ff-455c-9bf6-e69e1f6a8a8b/copy-manifests/0.log" Jul 16 11:13:13.429339 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:13.429311 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-tntpt_eb815613-68ff-455c-9bf6-e69e1f6a8a8b/manager/0.log" Jul 16 11:13:13.806664 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:13.806572 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" path="/var/lib/kubelet/pods/5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5/volumes" Jul 16 11:13:13.937319 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:13.937280 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54d48cd479-tzh2k_0ac9e004-d93d-41cc-82b3-819be495d5be/maas-api/0.log" Jul 16 11:13:15.074069 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.074038 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr_f80cc46e-073c-48cb-8cff-0b8fe5db5796/util/0.log" Jul 16 11:13:15.082059 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.082039 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr_f80cc46e-073c-48cb-8cff-0b8fe5db5796/pull/0.log" Jul 16 11:13:15.094878 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.094858 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr_f80cc46e-073c-48cb-8cff-0b8fe5db5796/extract/0.log" Jul 16 11:13:15.207872 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.207848 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7_460d2887-3a19-498b-a95d-e9839bf5c256/util/0.log" Jul 16 11:13:15.215724 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.215682 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7_460d2887-3a19-498b-a95d-e9839bf5c256/pull/0.log" Jul 16 11:13:15.223757 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.223735 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7_460d2887-3a19-498b-a95d-e9839bf5c256/extract/0.log" Jul 16 11:13:15.345798 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.345720 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5_209bd80e-ba3c-406d-92f8-27f1b6d506e6/util/0.log" Jul 16 11:13:15.353980 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.353952 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5_209bd80e-ba3c-406d-92f8-27f1b6d506e6/pull/0.log" Jul 16 11:13:15.361461 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.361443 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5_209bd80e-ba3c-406d-92f8-27f1b6d506e6/extract/0.log" Jul 16 11:13:15.477745 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.477721 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x_dac76e78-2e12-48cf-a721-5b91709d6136/util/0.log" Jul 16 11:13:15.485602 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.485584 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x_dac76e78-2e12-48cf-a721-5b91709d6136/pull/0.log" Jul 16 11:13:15.493288 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.493266 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x_dac76e78-2e12-48cf-a721-5b91709d6136/extract/0.log" Jul 16 11:13:15.618162 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.618078 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-564b4db568-jzcjj_be35bc62-d858-4792-ac1f-7e904eddaf28/authorino/0.log" Jul 16 11:13:15.740008 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.739979 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-s6ndq_c19d9c7e-db07-4b42-ac19-2932a6716ab3/manager/0.log" Jul 16 11:13:15.857231 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.857199 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-95984_081940f1-f71d-4157-92f8-9c6eadd3e93d/manager/0.log" Jul 16 11:13:15.970812 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:15.970737 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-ldstt_5b97797a-f035-4886-a2c3-579fc6482430/kuadrant-console-plugin/0.log" Jul 16 11:13:16.096439 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:16.096411 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-prcsn_6cb0bee4-8510-446a-a024-d09554bd1453/registry-server/0.log" Jul 16 11:13:16.397003 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:16.396975 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xrfw2_3bfad0fc-6892-4b7d-842d-d8c597ce1505/limitador/0.log" Jul 16 11:13:16.525869 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:16.525843 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9w7v4_6049c64a-a51d-4353-b311-dd4f62601d61/manager/0.log" Jul 16 11:13:17.252713 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:17.252662 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58946d4b5c-wcc4b_8e635e43-7c5c-403a-be45-836800d9fb89/kube-auth-proxy/0.log" Jul 16 11:13:17.394127 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:17.394096 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g82mq_e6bcaaaa-9f7e-4c89-8c94-c66486302286/istio-proxy/1.log" Jul 16 11:13:17.524964 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:17.524879 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-gvklw_9d86f4dc-ea88-407b-8a60-6fc5c1ec6451/payload-pre-processing/0.log" Jul 16 11:13:17.659466 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:17.659439 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6rzvw_464f26a1-df36-4845-996c-ab2ec952e8f8/payload-processing/0.log" Jul 16 11:13:18.372092 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:18.372059 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-4gtxq_f1bb8e14-c388-4669-8d21-a2a0e772e6b1/main/0.log" Jul 16 11:13:24.020964 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.020930 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2z85x/must-gather-wcnx8"] Jul 16 11:13:24.021374 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.021298 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" Jul 16 11:13:24.021374 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.021309 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" Jul 16 11:13:24.021374 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.021325 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" Jul 16 11:13:24.021374 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.021330 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" Jul 16 11:13:24.021374 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.021342 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44229895-c23c-4161-af67-3efb5ca90fa7" containerName="test-curl-7557-140051043545424" Jul 16 11:13:24.021374 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.021349 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="44229895-c23c-4161-af67-3efb5ca90fa7" containerName="test-curl-7557-140051043545424" Jul 16 11:13:24.021616 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.021402 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" Jul 16 11:13:24.021616 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.021410 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="44229895-c23c-4161-af67-3efb5ca90fa7" containerName="test-curl-7557-140051043545424" Jul 16 11:13:24.021616 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.021417 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d1bfb91-4497-4eab-b0d8-a5ac02ca59a5" containerName="istio-proxy" Jul 16 11:13:24.026326 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.026302 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2z85x/must-gather-wcnx8" Jul 16 11:13:24.029963 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.029933 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2z85x\"/\"openshift-service-ca.crt\"" Jul 16 11:13:24.030097 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.029937 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2z85x\"/\"default-dockercfg-xjzjv\"" Jul 16 11:13:24.030097 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.029950 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2z85x\"/\"kube-root-ca.crt\"" Jul 16 11:13:24.039038 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.039011 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2z85x/must-gather-wcnx8"] Jul 16 11:13:24.088345 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.088306 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4z8q\" (UniqueName: \"kubernetes.io/projected/44dffd60-195e-40c5-9a4f-c15c8ad1db17-kube-api-access-v4z8q\") pod \"must-gather-wcnx8\" (UID: \"44dffd60-195e-40c5-9a4f-c15c8ad1db17\") " pod="openshift-must-gather-2z85x/must-gather-wcnx8" Jul 16 11:13:24.088345 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.088345 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/44dffd60-195e-40c5-9a4f-c15c8ad1db17-must-gather-output\") pod \"must-gather-wcnx8\" (UID: \"44dffd60-195e-40c5-9a4f-c15c8ad1db17\") " pod="openshift-must-gather-2z85x/must-gather-wcnx8" Jul 16 11:13:24.189747 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.189682 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4z8q\" (UniqueName: \"kubernetes.io/projected/44dffd60-195e-40c5-9a4f-c15c8ad1db17-kube-api-access-v4z8q\") pod \"must-gather-wcnx8\" (UID: \"44dffd60-195e-40c5-9a4f-c15c8ad1db17\") " pod="openshift-must-gather-2z85x/must-gather-wcnx8" Jul 16 11:13:24.189747 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.189750 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/44dffd60-195e-40c5-9a4f-c15c8ad1db17-must-gather-output\") pod \"must-gather-wcnx8\" (UID: \"44dffd60-195e-40c5-9a4f-c15c8ad1db17\") " pod="openshift-must-gather-2z85x/must-gather-wcnx8" Jul 16 11:13:24.190079 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.190064 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/44dffd60-195e-40c5-9a4f-c15c8ad1db17-must-gather-output\") pod \"must-gather-wcnx8\" (UID: \"44dffd60-195e-40c5-9a4f-c15c8ad1db17\") " pod="openshift-must-gather-2z85x/must-gather-wcnx8" Jul 16 11:13:24.201155 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.201126 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4z8q\" (UniqueName: \"kubernetes.io/projected/44dffd60-195e-40c5-9a4f-c15c8ad1db17-kube-api-access-v4z8q\") pod \"must-gather-wcnx8\" (UID: \"44dffd60-195e-40c5-9a4f-c15c8ad1db17\") " pod="openshift-must-gather-2z85x/must-gather-wcnx8" Jul 16 11:13:24.336318 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.336282 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2z85x/must-gather-wcnx8" Jul 16 11:13:24.502619 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.502593 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2z85x/must-gather-wcnx8"] Jul 16 11:13:24.504295 ip-10-0-142-37 kubenswrapper[2581]: W0716 11:13:24.504269 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44dffd60_195e_40c5_9a4f_c15c8ad1db17.slice/crio-1214c0da6be2314d4792fbe816761ec8d007992505f72b967956468f370686e5 WatchSource:0}: Error finding container 1214c0da6be2314d4792fbe816761ec8d007992505f72b967956468f370686e5: Status 404 returned error can't find the container with id 1214c0da6be2314d4792fbe816761ec8d007992505f72b967956468f370686e5 Jul 16 11:13:24.976101 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:24.976063 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2z85x/must-gather-wcnx8" event={"ID":"44dffd60-195e-40c5-9a4f-c15c8ad1db17","Type":"ContainerStarted","Data":"1214c0da6be2314d4792fbe816761ec8d007992505f72b967956468f370686e5"} Jul 16 11:13:25.986524 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:25.986486 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2z85x/must-gather-wcnx8" event={"ID":"44dffd60-195e-40c5-9a4f-c15c8ad1db17","Type":"ContainerStarted","Data":"f73a9160fd4372fb33434cec8def6efa65eadf72ffcc2f6e9866622fe8d83342"} Jul 16 11:13:25.986979 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:25.986539 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2z85x/must-gather-wcnx8" event={"ID":"44dffd60-195e-40c5-9a4f-c15c8ad1db17","Type":"ContainerStarted","Data":"35aaab0a8c9bbf81444369b49bff3d38a862f1b0e8fa162c1c961b4a57a047a6"} Jul 16 11:13:26.005216 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:26.005162 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2z85x/must-gather-wcnx8" podStartSLOduration=2.178577748 podStartE2EDuration="3.005145033s" totalImagesPullingTime="826.567285ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:13:26.004568329 +0000 UTC m=+2460.683833309" watchObservedRunningTime="2026-07-16 11:13:26.005145033 +0000 UTC m=+2460.684410012" Jul 16 11:13:27.011615 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:27.011588 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-82c65_24b769d5-b98a-45d0-97fa-e67a68008449/global-pull-secret-syncer/0.log" Jul 16 11:13:27.345498 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:27.345439 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zs5vl_0e086493-640f-43e4-a8bf-cc5f03d52abe/konnectivity-agent/0.log" Jul 16 11:13:27.427993 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:27.427943 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-37.ec2.internal_26a59dee568ae6e70ff513d52676f317/haproxy/0.log" Jul 16 11:13:31.481889 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.481854 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr_f80cc46e-073c-48cb-8cff-0b8fe5db5796/extract/0.log" Jul 16 11:13:31.510939 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.510906 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr_f80cc46e-073c-48cb-8cff-0b8fe5db5796/util/0.log" Jul 16 11:13:31.536153 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.536120 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592rtxr_f80cc46e-073c-48cb-8cff-0b8fe5db5796/pull/0.log" Jul 16 11:13:31.571991 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.571959 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7_460d2887-3a19-498b-a95d-e9839bf5c256/extract/0.log" Jul 16 11:13:31.597095 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.597013 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7_460d2887-3a19-498b-a95d-e9839bf5c256/util/0.log" Jul 16 11:13:31.624040 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.623978 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lqbz7_460d2887-3a19-498b-a95d-e9839bf5c256/pull/0.log" Jul 16 11:13:31.656935 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.656904 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5_209bd80e-ba3c-406d-92f8-27f1b6d506e6/extract/0.log" Jul 16 11:13:31.684452 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.684425 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5_209bd80e-ba3c-406d-92f8-27f1b6d506e6/util/0.log" Jul 16 11:13:31.712862 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.712819 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l8vk5_209bd80e-ba3c-406d-92f8-27f1b6d506e6/pull/0.log" Jul 16 11:13:31.762954 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.762871 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x_dac76e78-2e12-48cf-a721-5b91709d6136/extract/0.log" Jul 16 11:13:31.799639 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.799602 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x_dac76e78-2e12-48cf-a721-5b91709d6136/util/0.log" Jul 16 11:13:31.840870 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:31.840846 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jbn6x_dac76e78-2e12-48cf-a721-5b91709d6136/pull/0.log" Jul 16 11:13:32.400067 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:32.400035 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-564b4db568-jzcjj_be35bc62-d858-4792-ac1f-7e904eddaf28/authorino/0.log" Jul 16 11:13:32.438069 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:32.438040 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-s6ndq_c19d9c7e-db07-4b42-ac19-2932a6716ab3/manager/0.log" Jul 16 11:13:32.476327 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:32.476300 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-95984_081940f1-f71d-4157-92f8-9c6eadd3e93d/manager/0.log" Jul 16 11:13:32.516404 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:32.516380 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-ldstt_5b97797a-f035-4886-a2c3-579fc6482430/kuadrant-console-plugin/0.log" Jul 16 11:13:32.566205 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:32.566160 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-prcsn_6cb0bee4-8510-446a-a024-d09554bd1453/registry-server/0.log" Jul 16 11:13:32.807323 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:32.807239 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xrfw2_3bfad0fc-6892-4b7d-842d-d8c597ce1505/limitador/0.log" Jul 16 11:13:32.912170 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:32.912134 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9w7v4_6049c64a-a51d-4353-b311-dd4f62601d61/manager/0.log" Jul 16 11:13:34.699905 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:34.699881 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb84ae97-4f67-41f0-920f-183be30c6d39/alertmanager/0.log" Jul 16 11:13:34.788127 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:34.788090 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb84ae97-4f67-41f0-920f-183be30c6d39/config-reloader/0.log" Jul 16 11:13:34.870900 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:34.870868 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb84ae97-4f67-41f0-920f-183be30c6d39/kube-rbac-proxy-web/0.log" Jul 16 11:13:34.940958 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:34.940928 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb84ae97-4f67-41f0-920f-183be30c6d39/kube-rbac-proxy/0.log" Jul 16 11:13:34.978346 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:34.978262 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb84ae97-4f67-41f0-920f-183be30c6d39/kube-rbac-proxy-metric/0.log" Jul 16 11:13:35.002279 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:35.002255 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb84ae97-4f67-41f0-920f-183be30c6d39/prom-label-proxy/0.log" Jul 16 11:13:35.030027 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:35.029990 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb84ae97-4f67-41f0-920f-183be30c6d39/init-config-reloader/0.log" Jul 16 11:13:35.293352 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:35.293275 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2t594_7824e80e-98c8-4eee-b7a5-b7ec0ea48103/node-exporter/0.log" Jul 16 11:13:35.321756 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:35.321729 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2t594_7824e80e-98c8-4eee-b7a5-b7ec0ea48103/kube-rbac-proxy/0.log" Jul 16 11:13:35.369707 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:35.369665 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2t594_7824e80e-98c8-4eee-b7a5-b7ec0ea48103/init-textfile/0.log" Jul 16 11:13:37.225255 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.225224 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt"] Jul 16 11:13:37.230857 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.230834 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.241039 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.241006 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt"] Jul 16 11:13:37.334039 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.333995 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-lib-modules\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.334338 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.334316 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-podres\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.334619 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.334596 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsmgn\" (UniqueName: \"kubernetes.io/projected/6b785581-7948-4b67-b766-7b83e7d409c3-kube-api-access-jsmgn\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.334873 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.334854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-proc\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.335058 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.335041 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-sys\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.436742 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.436680 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-proc\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.436941 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.436770 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-sys\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.436941 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.436795 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-proc\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.436941 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.436805 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-lib-modules\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.436941 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.436854 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-podres\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.436941 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.436901 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsmgn\" (UniqueName: \"kubernetes.io/projected/6b785581-7948-4b67-b766-7b83e7d409c3-kube-api-access-jsmgn\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.436941 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.436921 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-sys\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.436941 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.436929 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-lib-modules\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.437329 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.437152 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b785581-7948-4b67-b766-7b83e7d409c3-podres\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.447507 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.447462 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsmgn\" (UniqueName: \"kubernetes.io/projected/6b785581-7948-4b67-b766-7b83e7d409c3-kube-api-access-jsmgn\") pod \"perf-node-gather-daemonset-wztvt\" (UID: \"6b785581-7948-4b67-b766-7b83e7d409c3\") " pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.549154 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.549076 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:37.720531 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:37.720499 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt"] Jul 16 11:13:37.723659 ip-10-0-142-37 kubenswrapper[2581]: W0716 11:13:37.723618 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6b785581_7948_4b67_b766_7b83e7d409c3.slice/crio-07ffbe571770209a9e58caa7f4a69a63bb3ba472bb31d384a0aa9933fea49f18 WatchSource:0}: Error finding container 07ffbe571770209a9e58caa7f4a69a63bb3ba472bb31d384a0aa9933fea49f18: Status 404 returned error can't find the container with id 07ffbe571770209a9e58caa7f4a69a63bb3ba472bb31d384a0aa9933fea49f18 Jul 16 11:13:38.104266 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:38.104166 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" event={"ID":"6b785581-7948-4b67-b766-7b83e7d409c3","Type":"ContainerStarted","Data":"e081b01bcc9deb4baa739e5069ffe67bc5ea505b20290091c0924f1bbb47cbe5"} Jul 16 11:13:38.104266 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:38.104253 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:38.104481 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:38.104271 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" event={"ID":"6b785581-7948-4b67-b766-7b83e7d409c3","Type":"ContainerStarted","Data":"07ffbe571770209a9e58caa7f4a69a63bb3ba472bb31d384a0aa9933fea49f18"} Jul 16 11:13:38.135588 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:38.135527 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" podStartSLOduration=1.135506381 podStartE2EDuration="1.135506381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:13:38.132110358 +0000 UTC m=+2472.811375340" watchObservedRunningTime="2026-07-16 11:13:38.135506381 +0000 UTC m=+2472.814771369" Jul 16 11:13:38.619382 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:38.619349 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7487448f6b-2jtcs_54b45578-1fbb-4531-98f3-665a393abbdc/console/0.log" Jul 16 11:13:40.438208 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:40.438170 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n8m4p_a4d6b706-1a35-49b4-ae0d-a297613a7546/dns/0.log" Jul 16 11:13:40.460132 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:40.460086 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n8m4p_a4d6b706-1a35-49b4-ae0d-a297613a7546/kube-rbac-proxy/0.log" Jul 16 11:13:40.509299 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:40.509271 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mtklw_0cb30e43-e55f-4dec-af3f-39960a5946cb/dns-node-resolver/0.log" Jul 16 11:13:41.174845 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:41.174819 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jkm76_b115d8a4-0ad2-4dd2-9f96-7f05fb7b3880/node-ca/0.log" Jul 16 11:13:42.357084 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:42.357050 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58946d4b5c-wcc4b_8e635e43-7c5c-403a-be45-836800d9fb89/kube-auth-proxy/0.log" Jul 16 11:13:42.390736 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:42.390674 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g82mq_e6bcaaaa-9f7e-4c89-8c94-c66486302286/istio-proxy/0.log" Jul 16 11:13:42.453617 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:42.453582 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g82mq_e6bcaaaa-9f7e-4c89-8c94-c66486302286/istio-proxy/1.log" Jul 16 11:13:42.499494 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:42.499441 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-gvklw_9d86f4dc-ea88-407b-8a60-6fc5c1ec6451/payload-pre-processing/0.log" Jul 16 11:13:42.542659 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:42.542622 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6rzvw_464f26a1-df36-4845-996c-ab2ec952e8f8/payload-processing/0.log" Jul 16 11:13:43.325308 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:43.325271 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9gnph_7e39891c-14f3-4ac9-948d-dcc8ded4165f/serve-healthcheck-canary/0.log" Jul 16 11:13:43.939803 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:43.939766 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8ts8v_d675d0fa-bd74-4250-b4a8-989405a36e94/kube-rbac-proxy/0.log" Jul 16 11:13:43.963300 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:43.963275 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8ts8v_d675d0fa-bd74-4250-b4a8-989405a36e94/exporter/0.log" Jul 16 11:13:43.990244 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:43.990215 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8ts8v_d675d0fa-bd74-4250-b4a8-989405a36e94/extractor/0.log" Jul 16 11:13:44.128587 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:44.128556 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2z85x/perf-node-gather-daemonset-wztvt" Jul 16 11:13:46.685671 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:46.685638 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7rw8c_536d56ea-4060-4215-b36b-9b7e2373466c/manager/0.log" Jul 16 11:13:46.918007 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:46.917968 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tkmkm_8632f28b-52ee-4b02-9106-1bc9363c4387/manager/0.log" Jul 16 11:13:47.094771 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:47.094715 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c86f4fc58-7lwtw_8f34fe98-3d95-4f84-a011-087c61950369/manager/0.log" Jul 16 11:13:47.182829 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:47.182743 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5q7wq_99c1a236-aca2-4120-9a1d-473b2a4ef53e/server/0.log" Jul 16 11:13:47.333628 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:47.333587 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-d9mmp_08dc4380-9e09-4e3a-ab75-dd064a191ebe/manager/0.log" Jul 16 11:13:47.559397 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:47.559362 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-tntpt_eb815613-68ff-455c-9bf6-e69e1f6a8a8b/manager/0.log" Jul 16 11:13:47.634811 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:47.634778 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d8f7d9f46-tntpt_eb815613-68ff-455c-9bf6-e69e1f6a8a8b/copy-manifests/0.log" Jul 16 11:13:47.726984 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:47.726955 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58946d4b5c-wcc4b_8e635e43-7c5c-403a-be45-836800d9fb89/kube-auth-proxy/0.log" Jul 16 11:13:47.755094 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:47.755052 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g82mq_e6bcaaaa-9f7e-4c89-8c94-c66486302286/istio-proxy/0.log" Jul 16 11:13:47.819184 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:47.817803 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g82mq_e6bcaaaa-9f7e-4c89-8c94-c66486302286/istio-proxy/1.log" Jul 16 11:13:47.854682 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:47.854646 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-gvklw_9d86f4dc-ea88-407b-8a60-6fc5c1ec6451/payload-pre-processing/0.log" Jul 16 11:13:47.896230 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:47.896200 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6rzvw_464f26a1-df36-4845-996c-ab2ec952e8f8/payload-processing/0.log" Jul 16 11:13:49.281494 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:49.281410 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-fdhff_53339cd6-473d-456d-a76a-cddd1884cf64/openshift-lws-operator/0.log" Jul 16 11:13:49.356234 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:49.356203 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58946d4b5c-wcc4b_8e635e43-7c5c-403a-be45-836800d9fb89/kube-auth-proxy/0.log" Jul 16 11:13:49.399951 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:49.399923 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g82mq_e6bcaaaa-9f7e-4c89-8c94-c66486302286/istio-proxy/0.log" Jul 16 11:13:49.463025 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:49.462997 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g82mq_e6bcaaaa-9f7e-4c89-8c94-c66486302286/istio-proxy/1.log" Jul 16 11:13:49.499826 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:49.499795 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-gvklw_9d86f4dc-ea88-407b-8a60-6fc5c1ec6451/payload-pre-processing/0.log" Jul 16 11:13:49.533919 ip-10-0-142-37 kubenswrapper[2581]: I0716 11:13:49.533841 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6rzvw_464f26a1-df36-4845-996c-ab2ec952e8f8/payload-processing/0.log"