Jul 10 14:43:12.370996 ip-10-0-135-1 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 14:43:12.371009 ip-10-0-135-1 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 14:43:12.371019 ip-10-0-135-1 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 14:43:12.371341 ip-10-0-135-1 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 14:43:22.377027 ip-10-0-135-1 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 14:43:22.377042 ip-10-0-135-1 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c32643040abb4667a1f946edc2e31f1d -- Jul 10 14:45:19.602629 ip-10-0-135-1 systemd[1]: Starting Kubernetes Kubelet... Jul 10 14:45:20.015601 ip-10-0-135-1 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 14:45:20.015601 ip-10-0-135-1 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 14:45:20.015601 ip-10-0-135-1 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 14:45:20.015601 ip-10-0-135-1 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 14:45:20.015601 ip-10-0-135-1 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 14:45:20.016246 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.015666 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 14:45:20.019939 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019921 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:45:20.019939 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019937 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:45:20.019939 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019940 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019945 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019948 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019951 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019954 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019957 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019960 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019963 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019966 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019968 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019971 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019973 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019976 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019978 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019981 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019983 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019986 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019988 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019993 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019995 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:45:20.020035 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.019998 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020000 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020003 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020006 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020010 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020013 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020016 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020019 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020022 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020024 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020027 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020029 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020032 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020034 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020038 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020041 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020044 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020047 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020049 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020052 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:45:20.020612 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020054 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020057 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020059 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020062 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020064 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020067 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020071 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020076 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020079 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020082 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020085 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020089 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020092 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020095 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020097 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020100 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020102 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020105 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020107 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020110 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:45:20.021281 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020113 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020115 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020118 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020120 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020123 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020126 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020129 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020131 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020135 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020138 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020140 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020143 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020146 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020148 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020151 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020154 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020156 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020159 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020162 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020164 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:45:20.021841 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020167 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020170 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020175 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020178 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020181 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020183 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020188 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020191 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020194 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020197 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020200 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020203 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020207 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020209 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020212 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020215 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:45:20.022364 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.020218 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:45:20.023389 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023373 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:45:20.023389 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023388 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023392 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023396 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023400 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023404 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023407 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023410 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023413 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023416 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023419 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023421 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023424 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023426 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023429 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023431 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023434 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023436 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023439 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023441 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:45:20.023449 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023444 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023446 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023449 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023452 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023454 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023457 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023464 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023467 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023470 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023472 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023474 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023477 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023479 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023482 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023484 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023487 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023489 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023492 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023494 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023497 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:45:20.023923 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023500 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023502 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023505 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023507 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023510 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023513 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023515 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023518 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023520 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023522 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023525 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023528 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023530 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023533 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023536 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023539 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023542 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023544 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023547 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023549 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:45:20.024419 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023552 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023554 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023557 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023560 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023562 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023565 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023567 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023570 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023573 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023575 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023578 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023580 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023582 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023585 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023589 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023593 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023595 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023598 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023600 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:45:20.025014 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023603 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023606 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023609 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023612 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023618 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023621 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023629 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023633 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023636 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023639 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023642 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023645 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023648 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023650 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023652 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023655 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023658 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023661 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023663 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:45:20.025530 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.023666 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024062 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024067 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024070 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024073 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024075 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024078 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024081 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024084 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024086 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024089 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024092 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024094 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024097 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024099 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024102 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024104 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024108 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024110 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024113 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:45:20.025981 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024116 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024118 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024121 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024124 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024126 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024129 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024132 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024135 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024137 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024139 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024142 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024144 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024147 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024149 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024152 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024155 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024157 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024159 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024162 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024165 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:45:20.026479 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024167 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024170 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024173 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024175 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024178 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024180 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024183 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024185 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024188 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024190 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024193 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024196 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024198 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024201 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024203 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024206 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024209 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024212 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024214 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:45:20.026991 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024217 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024219 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024222 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024224 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024227 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024229 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024231 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024235 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024240 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024244 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024247 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024253 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024256 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024259 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024261 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024264 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024267 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024270 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024273 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:45:20.027464 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024276 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024279 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024281 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024284 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024286 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024290 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024292 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024296 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024299 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024301 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024304 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024306 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024309 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024311 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024315 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024318 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024321 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024324 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024350 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:45:20.027955 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024353 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024356 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.024359 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024439 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024449 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024462 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024469 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024474 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024477 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024482 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024486 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024491 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024494 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024497 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024501 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024504 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024507 2562 flags.go:64] FLAG: --cgroup-root="" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024509 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024512 2562 flags.go:64] FLAG: --client-ca-file="" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024517 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024520 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024524 2562 flags.go:64] FLAG: --cluster-domain="" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024527 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 14:45:20.028433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024530 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024533 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024536 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024540 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024543 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024546 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024549 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024552 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024555 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024558 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024561 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024564 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024569 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024571 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024576 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024579 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024581 2562 flags.go:64] FLAG: --enable-server="true" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024584 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024589 2562 flags.go:64] FLAG: --event-burst="100" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024592 2562 flags.go:64] FLAG: --event-qps="50" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024598 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024602 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024605 2562 flags.go:64] FLAG: --eviction-hard="" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024609 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 14:45:20.028976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024612 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024615 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024618 2562 flags.go:64] FLAG: --eviction-soft="" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024621 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024624 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024627 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024630 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024633 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024636 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024638 2562 flags.go:64] FLAG: --feature-gates="" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024642 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024645 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024648 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024651 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024654 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024657 2562 flags.go:64] FLAG: --help="false" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024660 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024663 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024666 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024669 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024672 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024675 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024679 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 14:45:20.029584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024682 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024685 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024688 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024690 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024693 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024697 2562 flags.go:64] FLAG: --kube-reserved="" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024700 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024703 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024706 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024709 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024711 2562 flags.go:64] FLAG: --lock-file="" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024714 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024717 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024720 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024725 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024729 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024732 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024735 2562 flags.go:64] FLAG: --logging-format="text" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024738 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024741 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024744 2562 flags.go:64] FLAG: --manifest-url="" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024747 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024751 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024754 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024758 2562 flags.go:64] FLAG: --max-pods="110" Jul 10 14:45:20.030165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024761 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024764 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024767 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024770 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024773 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024776 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024779 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024792 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024795 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024798 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024801 2562 flags.go:64] FLAG: --pod-cidr="" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024803 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024811 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024813 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024816 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024819 2562 flags.go:64] FLAG: --port="10250" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024822 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024825 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f6dd049872280b4b" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024828 2562 flags.go:64] FLAG: --qos-reserved="" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024831 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024834 2562 flags.go:64] FLAG: --register-node="true" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024837 2562 flags.go:64] FLAG: --register-with-taints="" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024840 2562 flags.go:64] FLAG: --registry-burst="10" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024844 2562 flags.go:64] FLAG: --registry-qps="5" Jul 10 14:45:20.030794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024846 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024849 2562 flags.go:64] FLAG: --reserved-memory="" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024853 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024856 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024859 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024861 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024864 2562 flags.go:64] FLAG: --runonce="false" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024867 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024870 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024873 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024875 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024878 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024881 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024884 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024887 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024891 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024894 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024897 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024900 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024903 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024907 2562 flags.go:64] FLAG: --system-cgroups="" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024909 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024914 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024917 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024920 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 14:45:20.031454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024924 2562 flags.go:64] FLAG: --tls-min-version="" Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024926 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024929 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024932 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024935 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024939 2562 flags.go:64] FLAG: --v="2" Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024943 2562 flags.go:64] FLAG: --version="false" Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024951 2562 flags.go:64] FLAG: --vmodule="" Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024955 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.024958 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025075 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025078 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025081 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025084 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025087 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025090 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025092 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025095 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025097 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025100 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025102 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025105 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:45:20.032091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025109 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025111 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025114 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025116 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025119 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025123 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025125 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025128 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025132 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025135 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025138 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025141 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025143 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025146 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025149 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025151 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025154 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025157 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025160 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:45:20.032648 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025163 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025166 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025168 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025171 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025173 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025176 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025179 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025183 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025186 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025189 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025191 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025194 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025196 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025201 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025203 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025208 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025211 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025214 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025217 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:45:20.033095 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025220 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025222 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025225 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025227 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025230 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025232 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025235 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025238 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025240 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025243 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025245 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025248 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025250 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025253 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025256 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025259 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025261 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025264 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025266 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025269 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:45:20.033578 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025272 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025274 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025277 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025280 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025282 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025284 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025288 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025290 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025293 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025296 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025298 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025302 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025305 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025308 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025310 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025313 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025315 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025318 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025320 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025323 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:45:20.034060 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025325 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025348 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025351 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025354 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025357 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025360 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025363 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025366 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.025368 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.025895 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.032369 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.032385 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.032402 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.032411 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032468 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:45:20.034567 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032473 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032476 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032479 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032482 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032485 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032488 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032492 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032495 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032498 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032501 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032503 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032506 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032510 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032512 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032515 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032518 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032520 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032523 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032526 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032528 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:45:20.034930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032531 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032533 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032536 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032539 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032541 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032544 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032546 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032549 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032553 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032556 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032558 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032561 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032564 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032567 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032569 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032572 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032574 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032577 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032579 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032582 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:45:20.035437 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032584 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032587 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032589 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032592 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032594 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032597 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032599 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032602 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032605 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032607 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032610 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032612 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032615 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032617 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032620 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032622 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032625 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032628 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032630 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032633 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:45:20.035948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032636 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032639 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032643 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032646 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032649 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032652 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032656 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032659 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032662 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032665 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032667 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032669 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032672 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032674 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032677 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032680 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032683 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032685 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032688 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:45:20.036446 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032690 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032694 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032698 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032700 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032703 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032706 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032709 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032712 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032715 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032718 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032720 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032723 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032727 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032730 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032733 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032737 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032740 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032742 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:45:20.037017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032745 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.032750 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032859 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032864 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032867 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032869 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032872 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032875 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032877 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032880 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032883 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032885 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032889 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032892 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:45:20.037572 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032895 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032897 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032900 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032902 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032905 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032907 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032910 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032912 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032914 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032917 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032920 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032922 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032925 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032928 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032930 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032933 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032936 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032939 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032941 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032944 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:45:20.037957 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032946 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032949 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032951 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032954 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032956 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032959 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032961 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032964 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032968 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032971 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032974 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032977 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032980 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032982 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032985 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032987 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032990 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032993 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032995 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:45:20.038782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.032998 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033000 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033003 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033006 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033008 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033011 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033014 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033017 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033019 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033021 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033024 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033027 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033029 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033032 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033034 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033037 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033039 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033042 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033044 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033046 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033049 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:45:20.039260 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033051 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033054 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033056 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033059 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033061 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033064 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033066 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033069 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033073 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033076 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033079 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033081 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033084 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033087 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033090 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033093 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033095 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033098 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033101 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:45:20.039930 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033103 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033106 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033108 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033111 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033114 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033117 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033119 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:20.033121 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.033126 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.033257 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.035241 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.036658 2562 server.go:1019] "Starting client certificate rotation" Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.036775 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 14:45:20.040410 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.036829 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 14:45:20.040879 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.039767 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 14:45:20.041264 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.041146 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 14:45:20.052607 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.052589 2562 log.go:25] "Validated CRI v1 runtime API" Jul 10 14:45:20.055516 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.055499 2562 log.go:25] "Validated CRI v1 image API" Jul 10 14:45:20.056320 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.056294 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 14:45:20.059301 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.059275 2562 fs.go:135] Filesystem UUIDs: map[2790dddd-ba40-4034-a0b7-c946a043309c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b55e8163-b7b4-4f9f-8b6f-a646b2f6a641:/dev/nvme0n1p4] Jul 10 14:45:20.059386 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.059300 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 14:45:20.065027 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.064917 2562 manager.go:216] Machine: {Timestamp:2026-07-10 14:45:20.063857142 +0000 UTC m=+0.344371830 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093865 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23455fe337ef41c539fd4de1ab07db SystemUUID:ec23455f-e337-ef41-c539-fd4de1ab07db BootID:c3264304-0abb-4667-a1f9-46edc2e31f1d Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:45:65:55:d5:d5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:45:65:55:d5:d5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:5d:14:fd:55:72 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 14:45:20.065027 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.065024 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 14:45:20.065118 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.065104 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 14:45:20.066699 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.066678 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 14:45:20.066843 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.066702 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-1.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 14:45:20.067269 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.067252 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 14:45:20.067362 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.067325 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 14:45:20.067362 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.067361 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 14:45:20.067486 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.067370 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 14:45:20.068216 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.068205 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 14:45:20.068905 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.068895 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 14:45:20.071522 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.071509 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 10 14:45:20.071641 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.071632 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 14:45:20.071670 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.071665 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 10 14:45:20.071697 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.071677 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 14:45:20.071697 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.071688 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 14:45:20.071757 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.071699 2562 kubelet.go:398] "Adding apiserver pod source" Jul 10 14:45:20.071757 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.071709 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 14:45:20.072061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.072048 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 14:45:20.072097 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.072066 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 14:45:20.074569 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.074552 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 14:45:20.075964 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.075947 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 14:45:20.076009 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.075980 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 14:45:20.076695 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076681 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 14:45:20.076760 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076701 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 14:45:20.076760 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076707 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 14:45:20.076760 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076714 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 14:45:20.076760 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076724 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 14:45:20.076760 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076732 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 14:45:20.076948 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076938 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 14:45:20.076980 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076948 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 14:45:20.076980 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076955 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 14:45:20.076980 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076961 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 14:45:20.076980 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076975 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 14:45:20.077074 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.076983 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 14:45:20.078543 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.078533 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 14:45:20.078543 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.078544 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 14:45:20.081725 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.081709 2562 server.go:1268] "Started kubelet" Jul 10 14:45:20.081840 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.081784 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 14:45:20.081905 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.081855 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 14:45:20.081948 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.081934 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 14:45:20.086848 ip-10-0-135-1 systemd[1]: Started Kubernetes Kubelet. Jul 10 14:45:20.090094 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.090018 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 14:45:20.090201 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.090178 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-1.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 14:45:20.090262 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.090239 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 14:45:20.090628 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.090608 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 14:45:20.091708 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.091695 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 10 14:45:20.094018 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.093995 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 14:45:20.094098 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.093047 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-1.ec2.internal.18c0f4e5c2351f2f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-1.ec2.internal,UID:ip-10-0-135-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-1.ec2.internal,},FirstTimestamp:2026-07-10 14:45:20.081665839 +0000 UTC m=+0.362180528,LastTimestamp:2026-07-10 14:45:20.081665839 +0000 UTC m=+0.362180528,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-1.ec2.internal,}" Jul 10 14:45:20.094190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.094132 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ldhjk" Jul 10 14:45:20.094992 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.094975 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 14:45:20.095047 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.094979 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 14:45:20.095091 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.095068 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:20.095626 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.095609 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 14:45:20.095736 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.095636 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 14:45:20.095736 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.095611 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 14:45:20.095736 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.095708 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 10 14:45:20.095736 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.095717 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 10 14:45:20.096443 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.096424 2562 factory.go:55] Registering systemd factory Jul 10 14:45:20.096515 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.096447 2562 factory.go:223] Registration of the systemd container factory successfully Jul 10 14:45:20.096717 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.096702 2562 factory.go:153] Registering CRI-O factory Jul 10 14:45:20.096717 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.096717 2562 factory.go:223] Registration of the crio container factory successfully Jul 10 14:45:20.096816 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.096755 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 14:45:20.096816 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.096772 2562 factory.go:103] Registering Raw factory Jul 10 14:45:20.096816 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.096782 2562 manager.go:1196] Started watching for new ooms in manager Jul 10 14:45:20.097322 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.097307 2562 manager.go:349] Starting recovery of all containers Jul 10 14:45:20.102109 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.102081 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 14:45:20.102207 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.102087 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 14:45:20.102663 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.102643 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ldhjk" Jul 10 14:45:20.104061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.104027 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 14:45:20.108723 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.108706 2562 manager.go:354] Recovery completed Jul 10 14:45:20.110357 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.110307 2562 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 10 14:45:20.114637 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.114625 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:45:20.116495 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.116476 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:45:20.116568 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.116509 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:45:20.116568 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.116520 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:45:20.117128 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.117114 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 14:45:20.117128 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.117126 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:45:20.117233 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.117127 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 14:45:20.117233 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.117201 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 10 14:45:20.117905 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.117889 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:45:20.117971 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.117917 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:45:20.117971 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.117928 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:45:20.118468 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.118453 2562 policy_none.go:49] "None policy: Start" Jul 10 14:45:20.118503 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.118476 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 14:45:20.118503 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.118490 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 14:45:20.119017 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.118938 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-1.ec2.internal.18c0f4e5c44892df default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-1.ec2.internal,UID:ip-10-0-135-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-1.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-1.ec2.internal,},FirstTimestamp:2026-07-10 14:45:20.116495071 +0000 UTC m=+0.397009761,LastTimestamp:2026-07-10 14:45:20.116495071 +0000 UTC m=+0.397009761,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-1.ec2.internal,}" Jul 10 14:45:20.119530 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.119487 2562 policy_none.go:47] "Start" Jul 10 14:45:20.148883 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.148867 2562 manager.go:337] "Starting Device Plugin manager" Jul 10 14:45:20.148959 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.148921 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 14:45:20.148959 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.148934 2562 server.go:87] "Starting device plugin registration server" Jul 10 14:45:20.149192 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.149180 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 14:45:20.149248 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.149195 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 14:45:20.149303 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.149284 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 14:45:20.149389 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.149378 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 14:45:20.149442 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.149390 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 14:45:20.150138 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.150116 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 14:45:20.150227 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.150176 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:20.249592 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.249559 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:45:20.250599 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.250578 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:45:20.250664 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.250620 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:45:20.250664 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.250634 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:45:20.250664 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.250660 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.258931 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.258914 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.258983 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.258940 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-1.ec2.internal\": node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:20.272835 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.272780 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 14:45:20.272835 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.272821 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 14:45:20.272970 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.272841 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 14:45:20.272970 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.272924 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 14:45:20.274715 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.274696 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 14:45:20.276405 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.276387 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:20.373241 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.373200 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-1.ec2.internal"] Jul 10 14:45:20.373440 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.373293 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:45:20.374896 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.374878 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:45:20.374984 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.374916 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:45:20.374984 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.374931 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:45:20.376057 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.376041 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:45:20.376162 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.376146 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.376225 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.376176 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:45:20.376509 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.376494 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:20.376796 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.376783 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:45:20.376842 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.376802 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:45:20.376842 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.376811 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:45:20.376909 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.376840 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:45:20.376909 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.376862 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:45:20.376909 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.376871 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:45:20.377864 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.377851 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.377906 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.377876 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:45:20.378509 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.378491 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:45:20.378588 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.378520 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:45:20.378588 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.378529 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:45:20.397668 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.397643 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ce2037ca5160ce54131348f2295a5f9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal\" (UID: \"3ce2037ca5160ce54131348f2295a5f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.397759 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.397674 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ce2037ca5160ce54131348f2295a5f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal\" (UID: \"3ce2037ca5160ce54131348f2295a5f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.397759 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.397698 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/27fe56af2837f03ffde8b81c782d06c3-config\") pod \"kube-apiserver-proxy-ip-10-0-135-1.ec2.internal\" (UID: \"27fe56af2837f03ffde8b81c782d06c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.407994 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.407976 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-1.ec2.internal\" not found" node="ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.412476 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.412462 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-1.ec2.internal\" not found" node="ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.477494 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.477470 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:20.497859 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.497826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ce2037ca5160ce54131348f2295a5f9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal\" (UID: \"3ce2037ca5160ce54131348f2295a5f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.497982 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.497891 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ce2037ca5160ce54131348f2295a5f9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal\" (UID: \"3ce2037ca5160ce54131348f2295a5f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.497982 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.497920 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ce2037ca5160ce54131348f2295a5f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal\" (UID: \"3ce2037ca5160ce54131348f2295a5f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.497982 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.497943 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/27fe56af2837f03ffde8b81c782d06c3-config\") pod \"kube-apiserver-proxy-ip-10-0-135-1.ec2.internal\" (UID: \"27fe56af2837f03ffde8b81c782d06c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.498082 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.497986 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/27fe56af2837f03ffde8b81c782d06c3-config\") pod \"kube-apiserver-proxy-ip-10-0-135-1.ec2.internal\" (UID: \"27fe56af2837f03ffde8b81c782d06c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.498082 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.498015 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ce2037ca5160ce54131348f2295a5f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal\" (UID: \"3ce2037ca5160ce54131348f2295a5f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.578548 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.578472 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:20.679081 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.679041 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:20.710517 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.710488 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.715226 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.715208 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-1.ec2.internal" Jul 10 14:45:20.779587 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.779550 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:20.879921 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.879886 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:20.959303 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:20.959276 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 14:45:20.980628 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:20.980585 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:21.036403 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.036379 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 14:45:21.037051 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.036531 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 14:45:21.037051 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.036582 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 14:45:21.081382 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:21.081329 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:21.106148 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.106093 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 14:40:20 +0000 UTC" deadline="2028-01-16 01:02:38.402289167 +0000 UTC" Jul 10 14:45:21.106148 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.106141 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13306h17m17.29615261s" Jul 10 14:45:21.180931 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.180860 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 14:45:21.181864 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:21.181842 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-1.ec2.internal\" not found" Jul 10 14:45:21.234962 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.234931 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 14:45:21.282869 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:21.282831 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27fe56af2837f03ffde8b81c782d06c3.slice/crio-815130b8e4fb4e67cd7025c4cef453f6608384b11bba05c78ce0068819c3ba2d WatchSource:0}: Error finding container 815130b8e4fb4e67cd7025c4cef453f6608384b11bba05c78ce0068819c3ba2d: Status 404 returned error can't find the container with id 815130b8e4fb4e67cd7025c4cef453f6608384b11bba05c78ce0068819c3ba2d Jul 10 14:45:21.289026 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.289001 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 14:45:21.295723 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.295530 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" Jul 10 14:45:21.295828 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:21.295663 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ce2037ca5160ce54131348f2295a5f9.slice/crio-0eb7177cc8a0cf29d2cb19dc81fc180b84bcc6f816d1d9ccc490b5ca936bb622 WatchSource:0}: Error finding container 0eb7177cc8a0cf29d2cb19dc81fc180b84bcc6f816d1d9ccc490b5ca936bb622: Status 404 returned error can't find the container with id 0eb7177cc8a0cf29d2cb19dc81fc180b84bcc6f816d1d9ccc490b5ca936bb622 Jul 10 14:45:21.307540 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.307517 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 14:45:21.309102 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.309084 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-1.ec2.internal" Jul 10 14:45:21.318916 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:21.318894 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 14:45:22.072762 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.072729 2562 apiserver.go:52] "Watching apiserver" Jul 10 14:45:22.081577 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.081544 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 14:45:22.084161 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.083785 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2","openshift-dns/node-resolver-jgcmn","openshift-image-registry/node-ca-wkvd4","openshift-multus/multus-lsgpm","openshift-multus/network-metrics-daemon-rr8m4","openshift-network-operator/iptables-alerter-q2vs5","kube-system/global-pull-secret-syncer-dfm2x","openshift-cluster-node-tuning-operator/tuned-gz9cd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal","openshift-multus/multus-additional-cni-plugins-58qv9","openshift-network-diagnostics/network-check-target-g2z2c","openshift-ovn-kubernetes/ovnkube-node-zqbmp","kube-system/konnectivity-agent-thckg","kube-system/kube-apiserver-proxy-ip-10-0-135-1.ec2.internal"] Jul 10 14:45:22.085487 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.085465 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.086695 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.086434 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.088008 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.087985 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkvd4" Jul 10 14:45:22.089066 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.088776 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:22.089066 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.088873 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:22.090070 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.090047 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:22.090159 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.090122 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:22.090953 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.090931 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q2vs5" Jul 10 14:45:22.092133 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.092113 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.093424 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.093112 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jgcmn" Jul 10 14:45:22.094327 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.094303 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.096450 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.095822 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 14:45:22.097176 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.097152 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:22.097252 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.097219 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:22.098570 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.098547 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.099834 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.099813 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:22.102567 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.102543 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 14:45:22.102923 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.102905 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 14:45:22.103104 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.103089 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 14:45:22.103255 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.103241 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 14:45:22.103435 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.103398 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zlfcr\"" Jul 10 14:45:22.103694 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.103672 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 14:45:22.103755 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.103672 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rgzv8\"" Jul 10 14:45:22.103896 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.103875 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 14:45:22.103959 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.103949 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 14:45:22.109364 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.108844 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 14:45:22.109364 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.108975 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 14:45:22.109364 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.109141 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 14:45:22.109364 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.109186 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 14:45:22.109651 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.109612 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 14:45:22.110127 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.109730 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 14:45:22.110127 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.109766 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kkgzl\"" Jul 10 14:45:22.110127 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.109797 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 14:45:22.110127 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.109912 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 14:45:22.110127 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110055 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 14:45:22.110438 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110292 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 14:45:22.110438 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110306 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 14:45:22.110543 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110499 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 14:45:22.110543 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110517 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-kubernetes\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.110648 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110566 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-host\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.110648 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110606 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hcg98\"" Jul 10 14:45:22.110648 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110608 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e430080-473f-49a1-8290-dc04122b72c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.110648 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110525 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-64r7h\"" Jul 10 14:45:22.110855 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110710 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 14:45:22.110855 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-conf-dir\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.110955 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110853 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-daemon-config\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.110955 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110922 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9jz44\"" Jul 10 14:45:22.111060 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110925 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 14:45:22.111060 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110978 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 14:45:22.111060 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.110933 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-sysctl-d\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.111060 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tzrb\" (UniqueName: \"kubernetes.io/projected/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-kube-api-access-4tzrb\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.111311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111079 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-dbus\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:22.111311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-system-cni-dir\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.111311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111160 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-cni-binary-copy\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.111311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111194 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-var-lib-kubelet\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.111311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111226 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxtm4\" (UniqueName: \"kubernetes.io/projected/11d98938-d5e6-4a92-ac4b-fa8a333a09d5-kube-api-access-hxtm4\") pod \"node-resolver-jgcmn\" (UID: \"11d98938-d5e6-4a92-ac4b-fa8a333a09d5\") " pod="openshift-dns/node-resolver-jgcmn" Jul 10 14:45:22.111311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111251 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 14:45:22.111311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111261 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e430080-473f-49a1-8290-dc04122b72c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.111311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111293 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e0b6960f-1e07-4ece-a538-2ad030d6dab4-iptables-alerter-script\") pod \"iptables-alerter-q2vs5\" (UID: \"e0b6960f-1e07-4ece-a538-2ad030d6dab4\") " pod="openshift-network-operator/iptables-alerter-q2vs5" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111321 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-registration-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111351 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-tmp\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111384 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-os-release\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111420 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8bk4j\"" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111433 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-cnibin\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111475 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-run-multus-certs\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111502 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0b6960f-1e07-4ece-a538-2ad030d6dab4-host-slash\") pod \"iptables-alerter-q2vs5\" (UID: \"e0b6960f-1e07-4ece-a538-2ad030d6dab4\") " pod="openshift-network-operator/iptables-alerter-q2vs5" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-socket-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111564 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-device-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111594 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-sys-fs\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blqgh\" (UniqueName: \"kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh\") pod \"network-check-target-g2z2c\" (UID: \"cb1e9cb5-c361-4628-afd2-b341d97047a3\") " pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:22.111700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111686 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d82d7bc-fae9-4db0-941a-6f9988bfe13f-host\") pod \"node-ca-wkvd4\" (UID: \"5d82d7bc-fae9-4db0-941a-6f9988bfe13f\") " pod="openshift-image-registry/node-ca-wkvd4" Jul 10 14:45:22.112190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111747 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d82d7bc-fae9-4db0-941a-6f9988bfe13f-serviceca\") pod \"node-ca-wkvd4\" (UID: \"5d82d7bc-fae9-4db0-941a-6f9988bfe13f\") " pod="openshift-image-registry/node-ca-wkvd4" Jul 10 14:45:22.112190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111806 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-os-release\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.112190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111876 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:22.112190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111920 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-cnibin\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.112190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111955 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-etc-selinux\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.112190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.111986 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-sysctl-conf\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.112190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.112017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-systemd\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.112190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.112054 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jqbk\" (UniqueName: \"kubernetes.io/projected/5d82d7bc-fae9-4db0-941a-6f9988bfe13f-kube-api-access-4jqbk\") pod \"node-ca-wkvd4\" (UID: \"5d82d7bc-fae9-4db0-941a-6f9988bfe13f\") " pod="openshift-image-registry/node-ca-wkvd4" Jul 10 14:45:22.112190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.112098 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-var-lib-kubelet\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.112190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.112185 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-sysconfig\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.112550 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.112214 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-run\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.112550 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.112295 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-lib-modules\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.112550 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.112397 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-system-cni-dir\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.112550 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.112421 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-cni-dir\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.113704 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.113677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-var-lib-cni-multus\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.113792 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.113719 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-hostroot\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.113792 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.113746 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.113901 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.113811 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-tuned\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.113901 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.113834 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw9rv\" (UniqueName: \"kubernetes.io/projected/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-kube-api-access-gw9rv\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:22.113901 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.113858 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dttq2\" (UniqueName: \"kubernetes.io/projected/30bbb67d-5a2a-4c32-8d83-4d549011ec03-kube-api-access-dttq2\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.114407 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114388 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dcl74\"" Jul 10 14:45:22.114551 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114526 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-modprobe-d\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.114668 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11d98938-d5e6-4a92-ac4b-fa8a333a09d5-tmp-dir\") pod \"node-resolver-jgcmn\" (UID: \"11d98938-d5e6-4a92-ac4b-fa8a333a09d5\") " pod="openshift-dns/node-resolver-jgcmn" Jul 10 14:45:22.114668 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-run-k8s-cni-cncf-io\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.114668 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114615 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-run-netns\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.114668 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-etc-kubernetes\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.114668 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-kubelet-config\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:22.114931 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.114931 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114712 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e430080-473f-49a1-8290-dc04122b72c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.114931 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:22.114931 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114763 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11d98938-d5e6-4a92-ac4b-fa8a333a09d5-hosts-file\") pod \"node-resolver-jgcmn\" (UID: \"11d98938-d5e6-4a92-ac4b-fa8a333a09d5\") " pod="openshift-dns/node-resolver-jgcmn" Jul 10 14:45:22.114931 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-socket-dir-parent\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.114931 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114910 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbn44\" (UniqueName: \"kubernetes.io/projected/e0b6960f-1e07-4ece-a538-2ad030d6dab4-kube-api-access-kbn44\") pod \"iptables-alerter-q2vs5\" (UID: \"e0b6960f-1e07-4ece-a538-2ad030d6dab4\") " pod="openshift-network-operator/iptables-alerter-q2vs5" Jul 10 14:45:22.115262 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114939 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-var-lib-cni-bin\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.115262 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114965 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-sys\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.115262 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.114985 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjnkw\" (UniqueName: \"kubernetes.io/projected/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-kube-api-access-bjnkw\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.115262 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.115005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22rf5\" (UniqueName: \"kubernetes.io/projected/1e430080-473f-49a1-8290-dc04122b72c5-kube-api-access-22rf5\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.128350 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.127745 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 14:45:22.128350 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.128009 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 14:45:22.128350 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.128133 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 14:45:22.128350 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.128177 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 14:45:22.128350 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.128298 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qxw5g\"" Jul 10 14:45:22.138120 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.138080 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 14:45:22.159917 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.159894 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gfq58" Jul 10 14:45:22.169056 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.169033 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gfq58" Jul 10 14:45:22.197151 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.197099 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 14:45:22.215819 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.215759 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-cni-dir\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.215819 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.215807 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-var-lib-cni-multus\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.216061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.215831 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-hostroot\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.216061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.215857 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.216061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.215878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-tuned\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.216061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.215905 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-kubelet\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.216061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.215929 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e32e12c7-6c2e-455d-b703-6f648413a630-ovnkube-script-lib\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.216061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.215956 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw9rv\" (UniqueName: \"kubernetes.io/projected/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-kube-api-access-gw9rv\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:22.216061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.215982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dttq2\" (UniqueName: \"kubernetes.io/projected/30bbb67d-5a2a-4c32-8d83-4d549011ec03-kube-api-access-dttq2\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.216061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216006 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-modprobe-d\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.216061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216036 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11d98938-d5e6-4a92-ac4b-fa8a333a09d5-tmp-dir\") pod \"node-resolver-jgcmn\" (UID: \"11d98938-d5e6-4a92-ac4b-fa8a333a09d5\") " pod="openshift-dns/node-resolver-jgcmn" Jul 10 14:45:22.216061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216061 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-cni-bin\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-run-k8s-cni-cncf-io\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-run-netns\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216173 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-etc-kubernetes\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216200 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-kubelet-config\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216228 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216257 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e430080-473f-49a1-8290-dc04122b72c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216284 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-etc-openvswitch\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-node-log\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216338 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11d98938-d5e6-4a92-ac4b-fa8a333a09d5-hosts-file\") pod \"node-resolver-jgcmn\" (UID: \"11d98938-d5e6-4a92-ac4b-fa8a333a09d5\") " pod="openshift-dns/node-resolver-jgcmn" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216424 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-socket-dir-parent\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216452 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbn44\" (UniqueName: \"kubernetes.io/projected/e0b6960f-1e07-4ece-a538-2ad030d6dab4-kube-api-access-kbn44\") pod \"iptables-alerter-q2vs5\" (UID: \"e0b6960f-1e07-4ece-a538-2ad030d6dab4\") " pod="openshift-network-operator/iptables-alerter-q2vs5" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216480 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-var-lib-cni-bin\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-sys\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.216597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216535 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-hostroot\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjnkw\" (UniqueName: \"kubernetes.io/projected/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-kube-api-access-bjnkw\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216617 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-var-lib-cni-multus\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216641 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-var-lib-cni-bin\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216715 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11d98938-d5e6-4a92-ac4b-fa8a333a09d5-tmp-dir\") pod \"node-resolver-jgcmn\" (UID: \"11d98938-d5e6-4a92-ac4b-fa8a333a09d5\") " pod="openshift-dns/node-resolver-jgcmn" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216813 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-etc-kubernetes\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216837 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22rf5\" (UniqueName: \"kubernetes.io/projected/1e430080-473f-49a1-8290-dc04122b72c5-kube-api-access-22rf5\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216882 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-run-ovn-kubernetes\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216887 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-modprobe-d\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216929 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-cni-netd\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216960 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e32e12c7-6c2e-455d-b703-6f648413a630-ovnkube-config\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216979 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-kubelet-config\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.216985 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e32e12c7-6c2e-455d-b703-6f648413a630-env-overrides\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217018 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-kubernetes\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217032 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-sys\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217039 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-cni-dir\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217045 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-host\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.217444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217088 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-run-k8s-cni-cncf-io\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217137 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-socket-dir-parent\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217134 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.217164 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217185 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217200 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-kubernetes\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217234 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-run-netns\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.217260 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret podName:756c9aaf-f794-44f5-98fe-3ba47f92e0dd nodeName:}" failed. No retries permitted until 2026-07-10 14:45:22.717218404 +0000 UTC m=+2.997733102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret") pod "global-pull-secret-syncer-dfm2x" (UID: "756c9aaf-f794-44f5-98fe-3ba47f92e0dd") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217263 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-host\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e430080-473f-49a1-8290-dc04122b72c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217283 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11d98938-d5e6-4a92-ac4b-fa8a333a09d5-hosts-file\") pod \"node-resolver-jgcmn\" (UID: \"11d98938-d5e6-4a92-ac4b-fa8a333a09d5\") " pod="openshift-dns/node-resolver-jgcmn" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217483 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-run-openvswitch\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-conf-dir\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-daemon-config\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217621 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-conf-dir\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-sysctl-d\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217708 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e430080-473f-49a1-8290-dc04122b72c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.218351 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tzrb\" (UniqueName: \"kubernetes.io/projected/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-kube-api-access-4tzrb\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217766 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-dbus\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217781 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-sysctl-d\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217798 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-slash\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217803 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e430080-473f-49a1-8290-dc04122b72c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217838 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-run-netns\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217852 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-dbus\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217875 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-run-ovn\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217903 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-system-cni-dir\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217937 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-cni-binary-copy\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.217996 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-system-cni-dir\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-var-lib-kubelet\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218050 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxtm4\" (UniqueName: \"kubernetes.io/projected/11d98938-d5e6-4a92-ac4b-fa8a333a09d5-kube-api-access-hxtm4\") pod \"node-resolver-jgcmn\" (UID: \"11d98938-d5e6-4a92-ac4b-fa8a333a09d5\") " pod="openshift-dns/node-resolver-jgcmn" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218078 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-multus-daemon-config\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218128 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-var-lib-kubelet\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218129 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e430080-473f-49a1-8290-dc04122b72c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218166 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-log-socket\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.219228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218198 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67ff882f-6e91-4753-889f-2dbabaa9ac18-agent-certs\") pod \"konnectivity-agent-thckg\" (UID: \"67ff882f-6e91-4753-889f-2dbabaa9ac18\") " pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218419 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e0b6960f-1e07-4ece-a538-2ad030d6dab4-iptables-alerter-script\") pod \"iptables-alerter-q2vs5\" (UID: \"e0b6960f-1e07-4ece-a538-2ad030d6dab4\") " pod="openshift-network-operator/iptables-alerter-q2vs5" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-registration-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218475 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-tmp\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218485 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-cni-binary-copy\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-os-release\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218560 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-registration-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-run-systemd\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e32e12c7-6c2e-455d-b703-6f648413a630-ovn-node-metrics-cert\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-cnibin\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-run-multus-certs\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0b6960f-1e07-4ece-a538-2ad030d6dab4-host-slash\") pod \"iptables-alerter-q2vs5\" (UID: \"e0b6960f-1e07-4ece-a538-2ad030d6dab4\") " pod="openshift-network-operator/iptables-alerter-q2vs5" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218877 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-cnibin\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218891 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-os-release\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-socket-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218930 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-run-multus-certs\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218935 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0b6960f-1e07-4ece-a538-2ad030d6dab4-host-slash\") pod \"iptables-alerter-q2vs5\" (UID: \"e0b6960f-1e07-4ece-a538-2ad030d6dab4\") " pod="openshift-network-operator/iptables-alerter-q2vs5" Jul 10 14:45:22.220141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.218975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-device-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219007 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-sys-fs\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219036 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqgh\" (UniqueName: \"kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh\") pod \"network-check-target-g2z2c\" (UID: \"cb1e9cb5-c361-4628-afd2-b341d97047a3\") " pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219042 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-device-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219042 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-socket-dir\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219065 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d82d7bc-fae9-4db0-941a-6f9988bfe13f-host\") pod \"node-ca-wkvd4\" (UID: \"5d82d7bc-fae9-4db0-941a-6f9988bfe13f\") " pod="openshift-image-registry/node-ca-wkvd4" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219100 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-sys-fs\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219160 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d82d7bc-fae9-4db0-941a-6f9988bfe13f-host\") pod \"node-ca-wkvd4\" (UID: \"5d82d7bc-fae9-4db0-941a-6f9988bfe13f\") " pod="openshift-image-registry/node-ca-wkvd4" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219587 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e430080-473f-49a1-8290-dc04122b72c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219705 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e0b6960f-1e07-4ece-a538-2ad030d6dab4-iptables-alerter-script\") pod \"iptables-alerter-q2vs5\" (UID: \"e0b6960f-1e07-4ece-a538-2ad030d6dab4\") " pod="openshift-network-operator/iptables-alerter-q2vs5" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219095 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-var-lib-openvswitch\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67ff882f-6e91-4753-889f-2dbabaa9ac18-konnectivity-ca\") pod \"konnectivity-agent-thckg\" (UID: \"67ff882f-6e91-4753-889f-2dbabaa9ac18\") " pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219926 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d82d7bc-fae9-4db0-941a-6f9988bfe13f-serviceca\") pod \"node-ca-wkvd4\" (UID: \"5d82d7bc-fae9-4db0-941a-6f9988bfe13f\") " pod="openshift-image-registry/node-ca-wkvd4" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219954 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-os-release\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.219979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-cnibin\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220080 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-cnibin\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.221231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220118 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-os-release\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220159 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-etc-selinux\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.220167 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220190 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-sysctl-conf\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.220215 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs podName:47c939b7-87e0-4cc4-a214-d47ce0fb2ee2 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:22.720199021 +0000 UTC m=+3.000713713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs") pod "network-metrics-daemon-rr8m4" (UID: "47c939b7-87e0-4cc4-a214-d47ce0fb2ee2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220244 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-systemd-units\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220271 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klbz6\" (UniqueName: \"kubernetes.io/projected/e32e12c7-6c2e-455d-b703-6f648413a630-kube-api-access-klbz6\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220302 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-systemd\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220329 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jqbk\" (UniqueName: \"kubernetes.io/projected/5d82d7bc-fae9-4db0-941a-6f9988bfe13f-kube-api-access-4jqbk\") pod \"node-ca-wkvd4\" (UID: \"5d82d7bc-fae9-4db0-941a-6f9988bfe13f\") " pod="openshift-image-registry/node-ca-wkvd4" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220347 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30bbb67d-5a2a-4c32-8d83-4d549011ec03-etc-selinux\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-var-lib-kubelet\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220396 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-host-var-lib-kubelet\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-sysconfig\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220447 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d82d7bc-fae9-4db0-941a-6f9988bfe13f-serviceca\") pod \"node-ca-wkvd4\" (UID: \"5d82d7bc-fae9-4db0-941a-6f9988bfe13f\") " pod="openshift-image-registry/node-ca-wkvd4" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220455 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-sysconfig\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220452 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-run\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220492 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-run\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.222134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220303 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-sysctl-conf\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.222975 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-lib-modules\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.222975 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220538 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-system-cni-dir\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.222975 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220539 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-systemd\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.222975 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220667 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e430080-473f-49a1-8290-dc04122b72c5-system-cni-dir\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.222975 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.220672 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-lib-modules\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.223486 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.223456 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-etc-tuned\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.223579 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.223501 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-tmp\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.235060 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.235039 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 14:45:22.237122 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.236997 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxtm4\" (UniqueName: \"kubernetes.io/projected/11d98938-d5e6-4a92-ac4b-fa8a333a09d5-kube-api-access-hxtm4\") pod \"node-resolver-jgcmn\" (UID: \"11d98938-d5e6-4a92-ac4b-fa8a333a09d5\") " pod="openshift-dns/node-resolver-jgcmn" Jul 10 14:45:22.239433 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.239161 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:45:22.239433 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.239186 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:45:22.239433 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.239201 2562 projected.go:196] Error preparing data for projected volume kube-api-access-blqgh for pod openshift-network-diagnostics/network-check-target-g2z2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:22.239433 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.239263 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh podName:cb1e9cb5-c361-4628-afd2-b341d97047a3 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:22.739245488 +0000 UTC m=+3.019760182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-blqgh" (UniqueName: "kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh") pod "network-check-target-g2z2c" (UID: "cb1e9cb5-c361-4628-afd2-b341d97047a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:22.239433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.239298 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjnkw\" (UniqueName: \"kubernetes.io/projected/c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76-kube-api-access-bjnkw\") pod \"multus-lsgpm\" (UID: \"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76\") " pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.239841 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.239470 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22rf5\" (UniqueName: \"kubernetes.io/projected/1e430080-473f-49a1-8290-dc04122b72c5-kube-api-access-22rf5\") pod \"multus-additional-cni-plugins-58qv9\" (UID: \"1e430080-473f-49a1-8290-dc04122b72c5\") " pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.242275 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.242237 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw9rv\" (UniqueName: \"kubernetes.io/projected/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-kube-api-access-gw9rv\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:22.243479 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.243373 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbn44\" (UniqueName: \"kubernetes.io/projected/e0b6960f-1e07-4ece-a538-2ad030d6dab4-kube-api-access-kbn44\") pod \"iptables-alerter-q2vs5\" (UID: \"e0b6960f-1e07-4ece-a538-2ad030d6dab4\") " pod="openshift-network-operator/iptables-alerter-q2vs5" Jul 10 14:45:22.245619 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.245002 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jqbk\" (UniqueName: \"kubernetes.io/projected/5d82d7bc-fae9-4db0-941a-6f9988bfe13f-kube-api-access-4jqbk\") pod \"node-ca-wkvd4\" (UID: \"5d82d7bc-fae9-4db0-941a-6f9988bfe13f\") " pod="openshift-image-registry/node-ca-wkvd4" Jul 10 14:45:22.245619 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.245574 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tzrb\" (UniqueName: \"kubernetes.io/projected/bb2f42ca-48ba-4642-a8b6-85dca0fb25f9-kube-api-access-4tzrb\") pod \"tuned-gz9cd\" (UID: \"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.246504 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.246480 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dttq2\" (UniqueName: \"kubernetes.io/projected/30bbb67d-5a2a-4c32-8d83-4d549011ec03-kube-api-access-dttq2\") pod \"aws-ebs-csi-driver-node-wtpd2\" (UID: \"30bbb67d-5a2a-4c32-8d83-4d549011ec03\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.286393 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.286311 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" event={"ID":"3ce2037ca5160ce54131348f2295a5f9","Type":"ContainerStarted","Data":"0eb7177cc8a0cf29d2cb19dc81fc180b84bcc6f816d1d9ccc490b5ca936bb622"} Jul 10 14:45:22.289203 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.289117 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-1.ec2.internal" event={"ID":"27fe56af2837f03ffde8b81c782d06c3","Type":"ContainerStarted","Data":"815130b8e4fb4e67cd7025c4cef453f6608384b11bba05c78ce0068819c3ba2d"} Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-cni-netd\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321329 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e32e12c7-6c2e-455d-b703-6f648413a630-ovnkube-config\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e32e12c7-6c2e-455d-b703-6f648413a630-env-overrides\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321389 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-run-openvswitch\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321422 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-slash\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321443 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-run-netns\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321460 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-run-ovn\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-log-socket\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321501 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67ff882f-6e91-4753-889f-2dbabaa9ac18-agent-certs\") pod \"konnectivity-agent-thckg\" (UID: \"67ff882f-6e91-4753-889f-2dbabaa9ac18\") " pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-run-systemd\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e32e12c7-6c2e-455d-b703-6f648413a630-ovn-node-metrics-cert\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-var-lib-openvswitch\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67ff882f-6e91-4753-889f-2dbabaa9ac18-konnectivity-ca\") pod \"konnectivity-agent-thckg\" (UID: \"67ff882f-6e91-4753-889f-2dbabaa9ac18\") " pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321625 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-systemd-units\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321642 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klbz6\" (UniqueName: \"kubernetes.io/projected/e32e12c7-6c2e-455d-b703-6f648413a630-kube-api-access-klbz6\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321665 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-kubelet\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.324087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e32e12c7-6c2e-455d-b703-6f648413a630-ovnkube-script-lib\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321709 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-cni-bin\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321751 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-etc-openvswitch\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-node-log\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-run-ovn-kubernetes\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321895 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-run-ovn-kubernetes\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.321948 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-cni-netd\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.322461 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e32e12c7-6c2e-455d-b703-6f648413a630-ovnkube-config\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.322788 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e32e12c7-6c2e-455d-b703-6f648413a630-env-overrides\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.322848 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-run-openvswitch\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.322886 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-slash\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.322922 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-run-netns\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.322968 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-run-ovn\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.322990 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-log-socket\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.323431 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-cni-bin\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.323449 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-var-lib-openvswitch\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.323449 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-run-systemd\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.325103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.323492 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-kubelet\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.326003 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.323495 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-etc-openvswitch\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.326003 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.323557 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.326003 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.323601 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-systemd-units\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.326003 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.323647 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e32e12c7-6c2e-455d-b703-6f648413a630-node-log\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.326003 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.324007 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67ff882f-6e91-4753-889f-2dbabaa9ac18-konnectivity-ca\") pod \"konnectivity-agent-thckg\" (UID: \"67ff882f-6e91-4753-889f-2dbabaa9ac18\") " pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:22.326003 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.324227 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e32e12c7-6c2e-455d-b703-6f648413a630-ovnkube-script-lib\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.326810 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.326752 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67ff882f-6e91-4753-889f-2dbabaa9ac18-agent-certs\") pod \"konnectivity-agent-thckg\" (UID: \"67ff882f-6e91-4753-889f-2dbabaa9ac18\") " pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:22.326925 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.326843 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e32e12c7-6c2e-455d-b703-6f648413a630-ovn-node-metrics-cert\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.336867 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.336839 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klbz6\" (UniqueName: \"kubernetes.io/projected/e32e12c7-6c2e-455d-b703-6f648413a630-kube-api-access-klbz6\") pod \"ovnkube-node-zqbmp\" (UID: \"e32e12c7-6c2e-455d-b703-6f648413a630\") " pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.400094 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.399706 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lsgpm" Jul 10 14:45:22.411310 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:22.411259 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc83b077a_d8b6_4ba6_b5e3_c0b719e2cd76.slice/crio-aaff5dd3064f51d0b3c534042f53596e93bb021000d9abe71e6a8883202053ef WatchSource:0}: Error finding container aaff5dd3064f51d0b3c534042f53596e93bb021000d9abe71e6a8883202053ef: Status 404 returned error can't find the container with id aaff5dd3064f51d0b3c534042f53596e93bb021000d9abe71e6a8883202053ef Jul 10 14:45:22.412329 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.412092 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" Jul 10 14:45:22.422791 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.422766 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkvd4" Jul 10 14:45:22.424973 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:22.424938 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2f42ca_48ba_4642_a8b6_85dca0fb25f9.slice/crio-fcba6ffc85077702cad388af46244b8af18772048cab564d8014aaf431e03eb2 WatchSource:0}: Error finding container fcba6ffc85077702cad388af46244b8af18772048cab564d8014aaf431e03eb2: Status 404 returned error can't find the container with id fcba6ffc85077702cad388af46244b8af18772048cab564d8014aaf431e03eb2 Jul 10 14:45:22.431709 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.431689 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q2vs5" Jul 10 14:45:22.433023 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:22.432994 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d82d7bc_fae9_4db0_941a_6f9988bfe13f.slice/crio-74f266b0301bd1eb040d9aeaa7316e06ae0f6de7f67a2136ffef94098981000f WatchSource:0}: Error finding container 74f266b0301bd1eb040d9aeaa7316e06ae0f6de7f67a2136ffef94098981000f: Status 404 returned error can't find the container with id 74f266b0301bd1eb040d9aeaa7316e06ae0f6de7f67a2136ffef94098981000f Jul 10 14:45:22.440180 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.440158 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" Jul 10 14:45:22.442676 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:22.442646 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0b6960f_1e07_4ece_a538_2ad030d6dab4.slice/crio-beb5bdf0a4f28d6e13a885aa2400b53f979ec226790fae901fb16a1c50477316 WatchSource:0}: Error finding container beb5bdf0a4f28d6e13a885aa2400b53f979ec226790fae901fb16a1c50477316: Status 404 returned error can't find the container with id beb5bdf0a4f28d6e13a885aa2400b53f979ec226790fae901fb16a1c50477316 Jul 10 14:45:22.447213 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.447190 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jgcmn" Jul 10 14:45:22.457174 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:22.457121 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30bbb67d_5a2a_4c32_8d83_4d549011ec03.slice/crio-34c47835a71ecb365635ce16a1d1d9c360587272b663380e684e4a2f7150131d WatchSource:0}: Error finding container 34c47835a71ecb365635ce16a1d1d9c360587272b663380e684e4a2f7150131d: Status 404 returned error can't find the container with id 34c47835a71ecb365635ce16a1d1d9c360587272b663380e684e4a2f7150131d Jul 10 14:45:22.458133 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.458110 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-58qv9" Jul 10 14:45:22.461562 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:22.461514 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d98938_d5e6_4a92_ac4b_fa8a333a09d5.slice/crio-4f677ed00e1f235b5dfd84368d9a12c47102cc19501b552c057ac7c99830f637 WatchSource:0}: Error finding container 4f677ed00e1f235b5dfd84368d9a12c47102cc19501b552c057ac7c99830f637: Status 404 returned error can't find the container with id 4f677ed00e1f235b5dfd84368d9a12c47102cc19501b552c057ac7c99830f637 Jul 10 14:45:22.465232 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.465042 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:22.469175 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:22.469069 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e430080_473f_49a1_8290_dc04122b72c5.slice/crio-1dddb1834114cb4bb387a103fff806c19edf0143acbd565416e7a974aeac18cf WatchSource:0}: Error finding container 1dddb1834114cb4bb387a103fff806c19edf0143acbd565416e7a974aeac18cf: Status 404 returned error can't find the container with id 1dddb1834114cb4bb387a103fff806c19edf0143acbd565416e7a974aeac18cf Jul 10 14:45:22.472797 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.471851 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:22.724601 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.724503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:22.724601 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.724579 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:22.724807 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.724677 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:22.724807 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.724737 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret podName:756c9aaf-f794-44f5-98fe-3ba47f92e0dd nodeName:}" failed. No retries permitted until 2026-07-10 14:45:23.724723339 +0000 UTC m=+4.005238015 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret") pod "global-pull-secret-syncer-dfm2x" (UID: "756c9aaf-f794-44f5-98fe-3ba47f92e0dd") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:22.724807 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.724677 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:22.724807 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.724797 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs podName:47c939b7-87e0-4cc4-a214-d47ce0fb2ee2 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:23.724785335 +0000 UTC m=+4.005300015 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs") pod "network-metrics-daemon-rr8m4" (UID: "47c939b7-87e0-4cc4-a214-d47ce0fb2ee2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:22.825272 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:22.825240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqgh\" (UniqueName: \"kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh\") pod \"network-check-target-g2z2c\" (UID: \"cb1e9cb5-c361-4628-afd2-b341d97047a3\") " pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:22.825468 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.825431 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:45:22.825468 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.825458 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:45:22.825468 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.825468 2562 projected.go:196] Error preparing data for projected volume kube-api-access-blqgh for pod openshift-network-diagnostics/network-check-target-g2z2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:22.825596 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:22.825534 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh podName:cb1e9cb5-c361-4628-afd2-b341d97047a3 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:23.825508977 +0000 UTC m=+4.106023658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-blqgh" (UniqueName: "kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh") pod "network-check-target-g2z2c" (UID: "cb1e9cb5-c361-4628-afd2-b341d97047a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:23.172231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.172187 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 14:40:22 +0000 UTC" deadline="2028-01-14 03:47:12.409281926 +0000 UTC" Jul 10 14:45:23.172231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.172224 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13261h1m49.237061624s" Jul 10 14:45:23.273650 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.273567 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:23.273650 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.273623 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:23.273876 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:23.273723 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:23.273951 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:23.273863 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:23.296972 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.296932 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58qv9" event={"ID":"1e430080-473f-49a1-8290-dc04122b72c5","Type":"ContainerStarted","Data":"1dddb1834114cb4bb387a103fff806c19edf0143acbd565416e7a974aeac18cf"} Jul 10 14:45:23.299330 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.299292 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q2vs5" event={"ID":"e0b6960f-1e07-4ece-a538-2ad030d6dab4","Type":"ContainerStarted","Data":"beb5bdf0a4f28d6e13a885aa2400b53f979ec226790fae901fb16a1c50477316"} Jul 10 14:45:23.303622 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.303589 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkvd4" event={"ID":"5d82d7bc-fae9-4db0-941a-6f9988bfe13f","Type":"ContainerStarted","Data":"74f266b0301bd1eb040d9aeaa7316e06ae0f6de7f67a2136ffef94098981000f"} Jul 10 14:45:23.306358 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.306330 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lsgpm" event={"ID":"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76","Type":"ContainerStarted","Data":"aaff5dd3064f51d0b3c534042f53596e93bb021000d9abe71e6a8883202053ef"} Jul 10 14:45:23.308716 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.308685 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jgcmn" event={"ID":"11d98938-d5e6-4a92-ac4b-fa8a333a09d5","Type":"ContainerStarted","Data":"4f677ed00e1f235b5dfd84368d9a12c47102cc19501b552c057ac7c99830f637"} Jul 10 14:45:23.311266 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.311192 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" event={"ID":"30bbb67d-5a2a-4c32-8d83-4d549011ec03","Type":"ContainerStarted","Data":"34c47835a71ecb365635ce16a1d1d9c360587272b663380e684e4a2f7150131d"} Jul 10 14:45:23.315713 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.315685 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" event={"ID":"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9","Type":"ContainerStarted","Data":"fcba6ffc85077702cad388af46244b8af18772048cab564d8014aaf431e03eb2"} Jul 10 14:45:23.732225 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.732127 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:23.732393 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.732230 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:23.732473 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:23.732392 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:23.732473 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:23.732460 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs podName:47c939b7-87e0-4cc4-a214-d47ce0fb2ee2 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:25.732440507 +0000 UTC m=+6.012955197 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs") pod "network-metrics-daemon-rr8m4" (UID: "47c939b7-87e0-4cc4-a214-d47ce0fb2ee2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:23.732601 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:23.732528 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:23.732601 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:23.732562 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret podName:756c9aaf-f794-44f5-98fe-3ba47f92e0dd nodeName:}" failed. No retries permitted until 2026-07-10 14:45:25.732550775 +0000 UTC m=+6.013065468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret") pod "global-pull-secret-syncer-dfm2x" (UID: "756c9aaf-f794-44f5-98fe-3ba47f92e0dd") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:23.833531 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.833495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqgh\" (UniqueName: \"kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh\") pod \"network-check-target-g2z2c\" (UID: \"cb1e9cb5-c361-4628-afd2-b341d97047a3\") " pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:23.833703 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:23.833629 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:45:23.833703 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:23.833647 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:45:23.833703 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:23.833658 2562 projected.go:196] Error preparing data for projected volume kube-api-access-blqgh for pod openshift-network-diagnostics/network-check-target-g2z2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:23.833703 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:23.833704 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh podName:cb1e9cb5-c361-4628-afd2-b341d97047a3 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:25.833690451 +0000 UTC m=+6.114205143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-blqgh" (UniqueName: "kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh") pod "network-check-target-g2z2c" (UID: "cb1e9cb5-c361-4628-afd2-b341d97047a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:23.841353 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:23.841321 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 14:45:23.860571 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:45:23.860540 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ff882f_6e91_4753_889f_2dbabaa9ac18.slice/crio-27ad5e553712aa1a74069885370705785f3e1acf456f47e570e29ea8792231d3 WatchSource:0}: Error finding container 27ad5e553712aa1a74069885370705785f3e1acf456f47e570e29ea8792231d3: Status 404 returned error can't find the container with id 27ad5e553712aa1a74069885370705785f3e1acf456f47e570e29ea8792231d3 Jul 10 14:45:24.173336 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:24.173268 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 14:40:22 +0000 UTC" deadline="2028-03-23 12:08:47.057976896 +0000 UTC" Jul 10 14:45:24.173876 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:24.173332 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14925h23m22.884649333s" Jul 10 14:45:24.277027 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:24.275961 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:24.277027 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:24.276099 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:24.328548 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:24.328506 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" event={"ID":"e32e12c7-6c2e-455d-b703-6f648413a630","Type":"ContainerStarted","Data":"1bbb87d596b477f8f9360b9a09a92b2af78cb61ce89a85ebbb1e1f88dcd0ddb7"} Jul 10 14:45:24.335233 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:24.335148 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-1.ec2.internal" event={"ID":"27fe56af2837f03ffde8b81c782d06c3","Type":"ContainerStarted","Data":"0b80742c8a84c6198a02bd5622c6cf40be664cf6b965d30a638709bfcf5ed066"} Jul 10 14:45:24.337099 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:24.336988 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-thckg" event={"ID":"67ff882f-6e91-4753-889f-2dbabaa9ac18","Type":"ContainerStarted","Data":"27ad5e553712aa1a74069885370705785f3e1acf456f47e570e29ea8792231d3"} Jul 10 14:45:25.273861 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:25.273821 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:25.274375 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:25.273989 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:25.274559 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:25.274439 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:25.274559 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:25.274548 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:25.344651 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:25.344609 2562 generic.go:356] "Generic (PLEG): container finished" podID="3ce2037ca5160ce54131348f2295a5f9" containerID="b53c84d1628a694a76d9969d0fd37e96a7861cfb58dba843a7a0e10de4542ea2" exitCode=0 Jul 10 14:45:25.345737 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:25.345709 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" event={"ID":"3ce2037ca5160ce54131348f2295a5f9","Type":"ContainerDied","Data":"b53c84d1628a694a76d9969d0fd37e96a7861cfb58dba843a7a0e10de4542ea2"} Jul 10 14:45:25.367773 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:25.367711 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-1.ec2.internal" podStartSLOduration=4.367686788 podStartE2EDuration="4.367686788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:24.35414326 +0000 UTC m=+4.634657965" watchObservedRunningTime="2026-07-10 14:45:25.367686788 +0000 UTC m=+5.648201492" Jul 10 14:45:25.752308 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:25.751508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:25.752308 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:25.751575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:25.752308 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:25.751707 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:25.752308 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:25.751789 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret podName:756c9aaf-f794-44f5-98fe-3ba47f92e0dd nodeName:}" failed. No retries permitted until 2026-07-10 14:45:29.751768876 +0000 UTC m=+10.032283558 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret") pod "global-pull-secret-syncer-dfm2x" (UID: "756c9aaf-f794-44f5-98fe-3ba47f92e0dd") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:25.752308 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:25.752234 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:25.752308 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:25.752275 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs podName:47c939b7-87e0-4cc4-a214-d47ce0fb2ee2 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:29.752263376 +0000 UTC m=+10.032778054 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs") pod "network-metrics-daemon-rr8m4" (UID: "47c939b7-87e0-4cc4-a214-d47ce0fb2ee2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:25.852946 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:25.852780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqgh\" (UniqueName: \"kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh\") pod \"network-check-target-g2z2c\" (UID: \"cb1e9cb5-c361-4628-afd2-b341d97047a3\") " pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:25.853149 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:25.853064 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:45:25.853149 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:25.853094 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:45:25.853149 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:25.853111 2562 projected.go:196] Error preparing data for projected volume kube-api-access-blqgh for pod openshift-network-diagnostics/network-check-target-g2z2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:25.853322 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:25.853181 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh podName:cb1e9cb5-c361-4628-afd2-b341d97047a3 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:29.853159001 +0000 UTC m=+10.133673705 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-blqgh" (UniqueName: "kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh") pod "network-check-target-g2z2c" (UID: "cb1e9cb5-c361-4628-afd2-b341d97047a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:26.275948 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:26.275913 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:26.276483 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:26.276056 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:27.273609 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:27.273564 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:27.273814 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:27.273777 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:27.274262 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:27.274241 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:27.274373 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:27.274356 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:28.273090 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:28.273051 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:28.273484 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:28.273276 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:29.273732 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:29.273695 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:29.274228 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:29.273695 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:29.274228 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:29.273869 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:29.274228 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:29.273905 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:29.789445 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:29.789409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:29.789623 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:29.789482 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:29.789623 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:29.789600 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:29.789705 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:29.789648 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret podName:756c9aaf-f794-44f5-98fe-3ba47f92e0dd nodeName:}" failed. No retries permitted until 2026-07-10 14:45:37.789634293 +0000 UTC m=+18.070148970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret") pod "global-pull-secret-syncer-dfm2x" (UID: "756c9aaf-f794-44f5-98fe-3ba47f92e0dd") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:29.791224 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:29.791171 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:29.791366 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:29.791317 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs podName:47c939b7-87e0-4cc4-a214-d47ce0fb2ee2 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:37.7912968 +0000 UTC m=+18.071811490 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs") pod "network-metrics-daemon-rr8m4" (UID: "47c939b7-87e0-4cc4-a214-d47ce0fb2ee2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:29.890757 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:29.890709 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqgh\" (UniqueName: \"kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh\") pod \"network-check-target-g2z2c\" (UID: \"cb1e9cb5-c361-4628-afd2-b341d97047a3\") " pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:29.890965 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:29.890911 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:45:29.890965 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:29.890936 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:45:29.890965 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:29.890949 2562 projected.go:196] Error preparing data for projected volume kube-api-access-blqgh for pod openshift-network-diagnostics/network-check-target-g2z2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:29.891133 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:29.891011 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh podName:cb1e9cb5-c361-4628-afd2-b341d97047a3 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:37.890991957 +0000 UTC m=+18.171506649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-blqgh" (UniqueName: "kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh") pod "network-check-target-g2z2c" (UID: "cb1e9cb5-c361-4628-afd2-b341d97047a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:30.274626 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:30.274590 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:30.275084 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:30.274703 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:31.274323 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:31.273787 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:31.274323 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:31.273920 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:31.274323 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:31.274219 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:31.274323 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:31.274288 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:32.275492 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:32.275454 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:32.275881 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:32.275566 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:33.273580 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:33.273541 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:33.273771 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:33.273695 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:33.274119 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:33.274102 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:33.274202 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:33.274186 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:34.273661 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:34.273627 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:34.274007 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:34.273748 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:35.273771 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:35.273721 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:35.274231 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:35.273775 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:35.274231 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:35.273860 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:35.274231 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:35.273959 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:36.273517 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:36.273481 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:36.273696 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:36.273613 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:37.273404 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:37.273370 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:37.273404 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:37.273406 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:37.273935 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:37.273494 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:37.273935 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:37.273582 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:37.850543 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:37.850507 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:37.850735 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:37.850605 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:37.850735 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:37.850679 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:37.850735 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:37.850698 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:37.850862 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:37.850755 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret podName:756c9aaf-f794-44f5-98fe-3ba47f92e0dd nodeName:}" failed. No retries permitted until 2026-07-10 14:45:53.850733727 +0000 UTC m=+34.131248408 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret") pod "global-pull-secret-syncer-dfm2x" (UID: "756c9aaf-f794-44f5-98fe-3ba47f92e0dd") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:37.850862 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:37.850777 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs podName:47c939b7-87e0-4cc4-a214-d47ce0fb2ee2 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:53.85076671 +0000 UTC m=+34.131281392 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs") pod "network-metrics-daemon-rr8m4" (UID: "47c939b7-87e0-4cc4-a214-d47ce0fb2ee2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:37.951021 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:37.950963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqgh\" (UniqueName: \"kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh\") pod \"network-check-target-g2z2c\" (UID: \"cb1e9cb5-c361-4628-afd2-b341d97047a3\") " pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:37.951180 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:37.951131 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:45:37.951180 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:37.951159 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:45:37.951180 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:37.951174 2562 projected.go:196] Error preparing data for projected volume kube-api-access-blqgh for pod openshift-network-diagnostics/network-check-target-g2z2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:37.951312 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:37.951228 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh podName:cb1e9cb5-c361-4628-afd2-b341d97047a3 nodeName:}" failed. No retries permitted until 2026-07-10 14:45:53.95121437 +0000 UTC m=+34.231729051 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-blqgh" (UniqueName: "kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh") pod "network-check-target-g2z2c" (UID: "cb1e9cb5-c361-4628-afd2-b341d97047a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:38.273532 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:38.273500 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:38.273948 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:38.273632 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:39.273515 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:39.273472 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:39.273515 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:39.273507 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:39.274041 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:39.273614 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:39.274041 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:39.273761 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:40.274312 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:40.274276 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:40.274791 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:40.274386 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:41.273539 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:41.273499 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:41.273714 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:41.273627 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:41.273714 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:41.273698 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:41.273850 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:41.273815 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:42.273909 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:42.273879 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:42.274464 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:42.274004 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:43.273821 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.273627 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:43.274023 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.273643 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:43.274023 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:43.273937 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:43.274023 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:43.273979 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:43.386003 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.385923 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" event={"ID":"3ce2037ca5160ce54131348f2295a5f9","Type":"ContainerStarted","Data":"2b56966c8f0b3b4a7a2a4dca10a8b5e857bea2a7a43c844b0649b569afad53ba"} Jul 10 14:45:43.388304 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.388274 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e430080-473f-49a1-8290-dc04122b72c5" containerID="5dfafa6fd19e7481e8dcce8fb61cd30ecf915f99eb52a17c7c5fca271074126a" exitCode=0 Jul 10 14:45:43.388938 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.388912 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58qv9" event={"ID":"1e430080-473f-49a1-8290-dc04122b72c5","Type":"ContainerDied","Data":"5dfafa6fd19e7481e8dcce8fb61cd30ecf915f99eb52a17c7c5fca271074126a"} Jul 10 14:45:43.391139 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.391111 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkvd4" event={"ID":"5d82d7bc-fae9-4db0-941a-6f9988bfe13f","Type":"ContainerStarted","Data":"284543fed2759a740bea5f9e0c7e528fcb271f9ad88b40992a708632d56a6232"} Jul 10 14:45:43.392993 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.392965 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lsgpm" event={"ID":"c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76","Type":"ContainerStarted","Data":"c90607c06d086fa32eb30aae4e733c67377bafb78ceb913becb52caf926a88ca"} Jul 10 14:45:43.394467 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.394447 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-thckg" event={"ID":"67ff882f-6e91-4753-889f-2dbabaa9ac18","Type":"ContainerStarted","Data":"5b6207484c6ed90e1a471744e934ed62c2b3c915ed89c5ff80edd77c9bd5ee7e"} Jul 10 14:45:43.397576 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.397559 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 14:45:43.397966 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.397943 2562 generic.go:356] "Generic (PLEG): container finished" podID="e32e12c7-6c2e-455d-b703-6f648413a630" containerID="a602bbde111daecd1f6b70bd9907155de57d51bc64cc40a5c343ebb31823b363" exitCode=1 Jul 10 14:45:43.398071 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.397984 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" event={"ID":"e32e12c7-6c2e-455d-b703-6f648413a630","Type":"ContainerStarted","Data":"ea7b4b70afd4c9c9ef5bc9127a09e70988b10860a87db4a9be88bbff8074b45b"} Jul 10 14:45:43.398071 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.398053 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" event={"ID":"e32e12c7-6c2e-455d-b703-6f648413a630","Type":"ContainerStarted","Data":"3636116e107feccaefe35c7a08b1a079825999837b3b48cbd16f7250533aad46"} Jul 10 14:45:43.398178 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.398086 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" event={"ID":"e32e12c7-6c2e-455d-b703-6f648413a630","Type":"ContainerStarted","Data":"90aedab44e9ff2fa2bee03e2f98c6eec66fe0170af3b5d04697ee40e4889f4c1"} Jul 10 14:45:43.398178 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.398115 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" event={"ID":"e32e12c7-6c2e-455d-b703-6f648413a630","Type":"ContainerStarted","Data":"ad5129622f31f1939415531c448bd2636b4036bfd8d44cc7347cc9246f09f5ad"} Jul 10 14:45:43.398178 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.398158 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" event={"ID":"e32e12c7-6c2e-455d-b703-6f648413a630","Type":"ContainerDied","Data":"a602bbde111daecd1f6b70bd9907155de57d51bc64cc40a5c343ebb31823b363"} Jul 10 14:45:43.398325 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.398204 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" event={"ID":"e32e12c7-6c2e-455d-b703-6f648413a630","Type":"ContainerStarted","Data":"346a6d01e17ca0d94cac7a02d37737d12818e72bc99741200c42ebdce41ebb01"} Jul 10 14:45:43.399480 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.399456 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jgcmn" event={"ID":"11d98938-d5e6-4a92-ac4b-fa8a333a09d5","Type":"ContainerStarted","Data":"ce123ba5fbb06bfbfae9e8dc8a4f256508995b5fdf14003bdfe0a046729b5f70"} Jul 10 14:45:43.400903 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.400882 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" event={"ID":"30bbb67d-5a2a-4c32-8d83-4d549011ec03","Type":"ContainerStarted","Data":"a760364e61af74714debc6c5e8ffcc43cee07255b8bd3f82541c7cb8221067f4"} Jul 10 14:45:43.402537 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.402519 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" event={"ID":"bb2f42ca-48ba-4642-a8b6-85dca0fb25f9","Type":"ContainerStarted","Data":"5b8b4865050821800e59277b85508da723696ab8329272320c9cc6d3e3cd7d3c"} Jul 10 14:45:43.429385 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.429320 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-1.ec2.internal" podStartSLOduration=22.429307016 podStartE2EDuration="22.429307016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:43.407664027 +0000 UTC m=+23.688178724" watchObservedRunningTime="2026-07-10 14:45:43.429307016 +0000 UTC m=+23.709821714" Jul 10 14:45:43.429964 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.429932 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lsgpm" podStartSLOduration=3.279076775 podStartE2EDuration="23.429923987s" totalImagesPullingTime="20.150847212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:43.429536352 +0000 UTC m=+23.710051050" watchObservedRunningTime="2026-07-10 14:45:43.429923987 +0000 UTC m=+23.710438684" Jul 10 14:45:43.450703 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.450659 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gz9cd" podStartSLOduration=3.361614841 podStartE2EDuration="23.450645409s" totalImagesPullingTime="20.089030568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:43.449746878 +0000 UTC m=+23.730261576" watchObservedRunningTime="2026-07-10 14:45:43.450645409 +0000 UTC m=+23.731160107" Jul 10 14:45:43.532496 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.532439 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jgcmn" podStartSLOduration=2.491784641 podStartE2EDuration="22.532418312s" totalImagesPullingTime="20.040633671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:43.53157897 +0000 UTC m=+23.812093667" watchObservedRunningTime="2026-07-10 14:45:43.532418312 +0000 UTC m=+23.812933011" Jul 10 14:45:43.532638 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.532541 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wkvd4" podStartSLOduration=7.739121777 podStartE2EDuration="23.532535176s" totalImagesPullingTime="15.793413399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:43.502999274 +0000 UTC m=+23.783513973" watchObservedRunningTime="2026-07-10 14:45:43.532535176 +0000 UTC m=+23.813049874" Jul 10 14:45:43.553457 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:43.553400 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-thckg" podStartSLOduration=3.947111855 podStartE2EDuration="22.553380851s" totalImagesPullingTime="18.606268996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:43.553350484 +0000 UTC m=+23.833865174" watchObservedRunningTime="2026-07-10 14:45:43.553380851 +0000 UTC m=+23.833895549" Jul 10 14:45:44.041737 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:44.041580 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 14:45:44.164232 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:44.164089 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T14:45:44.041731878Z","UUID":"ec6499b1-ee7f-4cf7-81ed-46a98222d017","Handler":null,"Name":"","Endpoint":""} Jul 10 14:45:44.165909 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:44.165882 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 14:45:44.166058 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:44.165927 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 14:45:44.273429 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:44.273396 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:44.273617 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:44.273568 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:44.406360 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:44.406114 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q2vs5" event={"ID":"e0b6960f-1e07-4ece-a538-2ad030d6dab4","Type":"ContainerStarted","Data":"9a954f1cb17f26a39298560942a0b97373d9a1c02b7cb0e2a8a8dd27443429cc"} Jul 10 14:45:44.408455 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:44.408415 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" event={"ID":"30bbb67d-5a2a-4c32-8d83-4d549011ec03","Type":"ContainerStarted","Data":"414942433b5da29f03dbaa5d1fc4b4cafe11dda16d58d2a5e7334e338bcfe1b3"} Jul 10 14:45:44.423503 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:44.423420 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-q2vs5" podStartSLOduration=3.374317525 podStartE2EDuration="23.423405239s" totalImagesPullingTime="20.049087714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:44.422941757 +0000 UTC m=+24.703456456" watchObservedRunningTime="2026-07-10 14:45:44.423405239 +0000 UTC m=+24.703919936" Jul 10 14:45:44.537415 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:44.537386 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:44.538020 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:44.537989 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:45.274009 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:45.273974 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:45.274153 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:45.274020 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:45.274153 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:45.274111 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:45.274266 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:45.274223 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:45.412996 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:45.412960 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" event={"ID":"30bbb67d-5a2a-4c32-8d83-4d549011ec03","Type":"ContainerStarted","Data":"325e087231d8b8df888c8f2d9a52c7626e76df90a6152cd070e88a0011ecfc26"} Jul 10 14:45:45.413641 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:45.413615 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:45.414010 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:45.413992 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-thckg" Jul 10 14:45:45.436592 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:45.436535 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtpd2" podStartSLOduration=2.084773435 podStartE2EDuration="24.436516662s" totalImagesPullingTime="22.351743227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:45:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:45.435845517 +0000 UTC m=+25.716360225" watchObservedRunningTime="2026-07-10 14:45:45.436516662 +0000 UTC m=+25.717031361" Jul 10 14:45:46.277262 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:46.277226 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:46.277461 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:46.277342 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:46.419387 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:46.419319 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 14:45:46.419994 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:46.419860 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" event={"ID":"e32e12c7-6c2e-455d-b703-6f648413a630","Type":"ContainerStarted","Data":"a01d84e13c33d5ecb986e9d34ace729cf34001e102586065a80a447cdae6e1cc"} Jul 10 14:45:46.708246 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:46.708167 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jgcmn_11d98938-d5e6-4a92-ac4b-fa8a333a09d5/dns-node-resolver/0.log" Jul 10 14:45:47.273212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:47.273178 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:47.273429 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:47.273214 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:47.273429 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:47.273302 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:47.273552 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:47.273451 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:48.086935 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:48.086772 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkvd4_5d82d7bc-fae9-4db0-941a-6f9988bfe13f/node-ca/0.log" Jul 10 14:45:48.273464 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:48.273432 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:48.273622 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:48.273535 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:48.428195 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:48.428118 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 14:45:48.428532 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:48.428494 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" event={"ID":"e32e12c7-6c2e-455d-b703-6f648413a630","Type":"ContainerStarted","Data":"836b5aed70c2e13204bef1f5387c91c9cccc6f9a4085066cfccb8766b2dbcc28"} Jul 10 14:45:48.428730 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:48.428713 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:48.428934 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:48.428917 2562 scope.go:117] "RemoveContainer" containerID="a602bbde111daecd1f6b70bd9907155de57d51bc64cc40a5c343ebb31823b363" Jul 10 14:45:48.431944 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:48.431404 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e430080-473f-49a1-8290-dc04122b72c5" containerID="d06c7a3f3373ef3e91369a06e74ca2d9a3ca9297cc70c381acae0a737c03d5fd" exitCode=0 Jul 10 14:45:48.431944 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:48.431443 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58qv9" event={"ID":"1e430080-473f-49a1-8290-dc04122b72c5","Type":"ContainerDied","Data":"d06c7a3f3373ef3e91369a06e74ca2d9a3ca9297cc70c381acae0a737c03d5fd"} Jul 10 14:45:48.446960 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:48.446937 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:49.273451 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:49.273418 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:49.273908 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:49.273419 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:49.273908 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:49.273527 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:49.273908 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:49.273662 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:49.436772 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:49.436743 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 14:45:49.437105 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:49.437083 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" event={"ID":"e32e12c7-6c2e-455d-b703-6f648413a630","Type":"ContainerStarted","Data":"543409d4116f61303fb0ffed00d8a6e8ddcbc734ad1088cc9dfd9443ad9996e9"} Jul 10 14:45:49.437199 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:49.437185 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 14:45:49.437450 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:49.437434 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:49.453042 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:49.453023 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:49.506543 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:49.506492 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" podStartSLOduration=9.79136575 podStartE2EDuration="28.506478736s" totalImagesPullingTime="18.715112986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:49.474399333 +0000 UTC m=+29.754914031" watchObservedRunningTime="2026-07-10 14:45:49.506478736 +0000 UTC m=+29.786993434" Jul 10 14:45:50.274413 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:50.274227 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:50.274891 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:50.274477 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:50.440822 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:50.440786 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e430080-473f-49a1-8290-dc04122b72c5" containerID="9921e08ace77b500358fd87efd58185188ac51df7f406599ef12c7c1ec2f68ac" exitCode=0 Jul 10 14:45:50.441004 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:50.440874 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58qv9" event={"ID":"1e430080-473f-49a1-8290-dc04122b72c5","Type":"ContainerDied","Data":"9921e08ace77b500358fd87efd58185188ac51df7f406599ef12c7c1ec2f68ac"} Jul 10 14:45:50.441004 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:50.440924 2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 14:45:51.014935 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:51.014910 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:45:51.273609 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:51.273538 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:51.273756 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:51.273645 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:51.273756 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:51.273703 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:51.273828 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:51.273795 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:52.273966 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:52.273935 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:52.274357 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:52.274042 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:52.465574 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:52.465536 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" podUID="e32e12c7-6c2e-455d-b703-6f648413a630" containerName="ovnkube-controller" probeResult="failure" output="" Jul 10 14:45:53.273821 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:53.273782 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:53.274015 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:53.273914 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:53.274015 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:53.273981 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:53.274365 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:53.274098 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:53.871510 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:53.871479 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:53.871572 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:53.871562 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:53.871626 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:53.871606 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:53.871686 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:53.871675 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret podName:756c9aaf-f794-44f5-98fe-3ba47f92e0dd nodeName:}" failed. No retries permitted until 2026-07-10 14:46:25.871658937 +0000 UTC m=+66.152173634 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret") pod "global-pull-secret-syncer-dfm2x" (UID: "756c9aaf-f794-44f5-98fe-3ba47f92e0dd") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:45:53.871729 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:53.871685 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:53.871768 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:53.871744 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs podName:47c939b7-87e0-4cc4-a214-d47ce0fb2ee2 nodeName:}" failed. No retries permitted until 2026-07-10 14:46:25.871727822 +0000 UTC m=+66.152242513 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs") pod "network-metrics-daemon-rr8m4" (UID: "47c939b7-87e0-4cc4-a214-d47ce0fb2ee2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:45:53.972415 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:53.972377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqgh\" (UniqueName: \"kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh\") pod \"network-check-target-g2z2c\" (UID: \"cb1e9cb5-c361-4628-afd2-b341d97047a3\") " pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:53.972588 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:53.972539 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:45:53.972588 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:53.972557 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:45:53.972588 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:53.972567 2562 projected.go:196] Error preparing data for projected volume kube-api-access-blqgh for pod openshift-network-diagnostics/network-check-target-g2z2c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:53.972706 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:53.972617 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh podName:cb1e9cb5-c361-4628-afd2-b341d97047a3 nodeName:}" failed. No retries permitted until 2026-07-10 14:46:25.97260389 +0000 UTC m=+66.253118570 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-blqgh" (UniqueName: "kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh") pod "network-check-target-g2z2c" (UID: "cb1e9cb5-c361-4628-afd2-b341d97047a3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:45:54.273276 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:54.273240 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:54.273520 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:54.273382 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:54.453712 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:54.453678 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e430080-473f-49a1-8290-dc04122b72c5" containerID="311cdac5e8d77bbbe7ca9b2f0701741a0ac1a26cd360b5c9be7aa728ab0b13d3" exitCode=0 Jul 10 14:45:54.454139 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:54.453742 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58qv9" event={"ID":"1e430080-473f-49a1-8290-dc04122b72c5","Type":"ContainerDied","Data":"311cdac5e8d77bbbe7ca9b2f0701741a0ac1a26cd360b5c9be7aa728ab0b13d3"} Jul 10 14:45:55.273409 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:55.273375 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:55.273693 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:55.273496 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:55.273693 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:55.273551 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:55.273693 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:55.273661 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:56.274048 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:56.274009 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:56.274537 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:56.274149 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:57.273314 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:57.273271 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:57.273314 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:57.273313 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:57.273559 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:57.273426 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:57.273613 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:57.273579 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:58.273463 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:58.273430 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:45:58.273898 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:58.273559 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:45:59.273452 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:59.273418 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:45:59.273619 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:59.273531 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:45:59.273619 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:59.273591 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:45:59.273994 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:45:59.273695 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:45:59.467506 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:59.467473 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e430080-473f-49a1-8290-dc04122b72c5" containerID="be3a02bb02f31df449b4662c0e6fae49b26e55f3264b85749b8995191e3c4c57" exitCode=0 Jul 10 14:45:59.467506 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:45:59.467521 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58qv9" event={"ID":"1e430080-473f-49a1-8290-dc04122b72c5","Type":"ContainerDied","Data":"be3a02bb02f31df449b4662c0e6fae49b26e55f3264b85749b8995191e3c4c57"} Jul 10 14:46:00.274227 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:00.274194 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:00.274675 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:00.274274 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:46:00.472555 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:00.472520 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e430080-473f-49a1-8290-dc04122b72c5" containerID="02328a3cba93d8f2e67202c1aaf3a3c300553e0e599a15d24ce909f2ac7c4acc" exitCode=0 Jul 10 14:46:00.472682 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:00.472595 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58qv9" event={"ID":"1e430080-473f-49a1-8290-dc04122b72c5","Type":"ContainerDied","Data":"02328a3cba93d8f2e67202c1aaf3a3c300553e0e599a15d24ce909f2ac7c4acc"} Jul 10 14:46:01.273965 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:01.273938 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:01.274140 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:01.274040 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:46:01.274140 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:01.274097 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:01.274216 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:01.274199 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:46:01.477763 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:01.477730 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58qv9" event={"ID":"1e430080-473f-49a1-8290-dc04122b72c5","Type":"ContainerStarted","Data":"fa18ab9ac3bae1a67841a2ef9d4ca7846b63d2b8937234394f99956d5b19ed20"} Jul 10 14:46:01.506299 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:01.506253 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-58qv9" podStartSLOduration=7.07599466 podStartE2EDuration="40.506237315s" totalImagesPullingTime="33.430242655s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:45:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:01.504883888 +0000 UTC m=+41.785398599" watchObservedRunningTime="2026-07-10 14:46:01.506237315 +0000 UTC m=+41.786752012" Jul 10 14:46:02.273120 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:02.273086 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:02.273361 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:02.273221 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:46:03.274082 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:03.274042 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:03.274584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:03.274042 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:03.274584 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:03.274183 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:46:03.274584 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:03.274218 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:46:04.273615 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:04.273579 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:04.273762 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:04.273732 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:46:05.273590 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:05.273558 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:05.273972 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:05.273675 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:46:05.273972 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:05.273731 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:05.273972 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:05.273821 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:46:06.273206 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:06.273175 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:06.273491 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:06.273279 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:46:07.273739 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:07.273700 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:07.274138 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:07.273748 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:07.274138 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:07.273818 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:46:07.274138 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:07.273880 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:46:08.273773 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:08.273743 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:08.274159 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:08.273837 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:46:09.273070 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:09.273036 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:09.273070 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:09.273060 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:09.273263 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:09.273153 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:46:09.273300 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:09.273275 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:46:10.238384 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:10.238175 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g2z2c"] Jul 10 14:46:10.238940 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:10.238450 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:10.238940 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:10.238553 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:46:10.241410 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:10.241380 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dfm2x"] Jul 10 14:46:10.241575 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:10.241499 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:10.241653 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:10.241612 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:46:10.242073 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:10.242046 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rr8m4"] Jul 10 14:46:10.242189 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:10.242146 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:10.242278 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:10.242252 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:46:12.276252 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:12.276224 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:12.276682 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:12.276313 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:12.276682 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:12.276323 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:46:12.276682 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:12.276358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:12.276682 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:12.276424 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:46:12.276682 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:12.276507 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:46:14.273692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.273608 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:14.274102 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.273609 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:14.274102 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:14.273737 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rr8m4" podUID="47c939b7-87e0-4cc4-a214-d47ce0fb2ee2" Jul 10 14:46:14.274102 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.273608 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:14.274102 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:14.273825 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfm2x" podUID="756c9aaf-f794-44f5-98fe-3ba47f92e0dd" Jul 10 14:46:14.274102 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:14.273903 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g2z2c" podUID="cb1e9cb5-c361-4628-afd2-b341d97047a3" Jul 10 14:46:14.548097 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.548027 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-1.ec2.internal" event="NodeReady" Jul 10 14:46:14.548229 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.548144 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 14:46:14.601399 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.601368 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wp7kd"] Jul 10 14:46:14.606174 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.606154 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hhcdc"] Jul 10 14:46:14.606295 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.606278 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wp7kd" Jul 10 14:46:14.608477 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.608459 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 14:46:14.608654 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.608637 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 14:46:14.608863 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.608836 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 14:46:14.609000 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.608983 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zqj22\"" Jul 10 14:46:14.616402 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.616381 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wp7kd"] Jul 10 14:46:14.616491 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.616482 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.618705 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.618688 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 14:46:14.618906 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.618892 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 14:46:14.618989 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.618910 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 14:46:14.619169 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.619154 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dx9zp\"" Jul 10 14:46:14.619666 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.619651 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 14:46:14.623474 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.623451 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-znj5q"] Jul 10 14:46:14.625896 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.625878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn5ks\" (UniqueName: \"kubernetes.io/projected/e5c6f071-e790-4ccf-909f-131a20bff3ec-kube-api-access-cn5ks\") pod \"ingress-canary-wp7kd\" (UID: \"e5c6f071-e790-4ccf-909f-131a20bff3ec\") " pod="openshift-ingress-canary/ingress-canary-wp7kd" Jul 10 14:46:14.625972 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.625910 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5c6f071-e790-4ccf-909f-131a20bff3ec-cert\") pod \"ingress-canary-wp7kd\" (UID: \"e5c6f071-e790-4ccf-909f-131a20bff3ec\") " pod="openshift-ingress-canary/ingress-canary-wp7kd" Jul 10 14:46:14.636465 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.636440 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hhcdc"] Jul 10 14:46:14.636635 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.636612 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.639703 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.639685 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 14:46:14.639703 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.639693 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 14:46:14.639821 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.639713 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sqvz7\"" Jul 10 14:46:14.642708 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.642692 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-znj5q"] Jul 10 14:46:14.726581 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.726546 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-crio-socket\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.726581 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.726581 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.726808 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.726606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5wt9\" (UniqueName: \"kubernetes.io/projected/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-kube-api-access-q5wt9\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.726808 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.726622 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/475e4f82-6ecd-47d9-88e1-6901ff54ba92-config-volume\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.726808 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.726688 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-data-volume\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.726808 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.726733 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/475e4f82-6ecd-47d9-88e1-6901ff54ba92-tmp-dir\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.726808 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.726775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5ks\" (UniqueName: \"kubernetes.io/projected/e5c6f071-e790-4ccf-909f-131a20bff3ec-kube-api-access-cn5ks\") pod \"ingress-canary-wp7kd\" (UID: \"e5c6f071-e790-4ccf-909f-131a20bff3ec\") " pod="openshift-ingress-canary/ingress-canary-wp7kd" Jul 10 14:46:14.727019 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.726816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5c6f071-e790-4ccf-909f-131a20bff3ec-cert\") pod \"ingress-canary-wp7kd\" (UID: \"e5c6f071-e790-4ccf-909f-131a20bff3ec\") " pod="openshift-ingress-canary/ingress-canary-wp7kd" Jul 10 14:46:14.727019 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.726843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.727019 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.726868 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/475e4f82-6ecd-47d9-88e1-6901ff54ba92-metrics-tls\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.727019 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.726891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk97m\" (UniqueName: \"kubernetes.io/projected/475e4f82-6ecd-47d9-88e1-6901ff54ba92-kube-api-access-hk97m\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.731024 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.730998 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5c6f071-e790-4ccf-909f-131a20bff3ec-cert\") pod \"ingress-canary-wp7kd\" (UID: \"e5c6f071-e790-4ccf-909f-131a20bff3ec\") " pod="openshift-ingress-canary/ingress-canary-wp7kd" Jul 10 14:46:14.734908 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.734890 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn5ks\" (UniqueName: \"kubernetes.io/projected/e5c6f071-e790-4ccf-909f-131a20bff3ec-kube-api-access-cn5ks\") pod \"ingress-canary-wp7kd\" (UID: \"e5c6f071-e790-4ccf-909f-131a20bff3ec\") " pod="openshift-ingress-canary/ingress-canary-wp7kd" Jul 10 14:46:14.827918 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.827857 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-data-volume\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.827918 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.827890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/475e4f82-6ecd-47d9-88e1-6901ff54ba92-tmp-dir\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.828084 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.827924 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.828084 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.827941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/475e4f82-6ecd-47d9-88e1-6901ff54ba92-metrics-tls\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.828084 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.827955 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk97m\" (UniqueName: \"kubernetes.io/projected/475e4f82-6ecd-47d9-88e1-6901ff54ba92-kube-api-access-hk97m\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.828084 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.827978 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-crio-socket\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.828084 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.827997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.828084 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.828034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5wt9\" (UniqueName: \"kubernetes.io/projected/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-kube-api-access-q5wt9\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.828084 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.828054 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/475e4f82-6ecd-47d9-88e1-6901ff54ba92-config-volume\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.828433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.828167 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-crio-socket\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.828433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.828286 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/475e4f82-6ecd-47d9-88e1-6901ff54ba92-tmp-dir\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.828433 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.828299 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-data-volume\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.828585 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.828567 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.828754 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.828735 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/475e4f82-6ecd-47d9-88e1-6901ff54ba92-config-volume\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.830450 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.830427 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/475e4f82-6ecd-47d9-88e1-6901ff54ba92-metrics-tls\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.830658 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.830640 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.836856 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.836826 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5wt9\" (UniqueName: \"kubernetes.io/projected/fd9bc4d0-177d-465c-8b8f-60daebc95bc6-kube-api-access-q5wt9\") pod \"insights-runtime-extractor-hhcdc\" (UID: \"fd9bc4d0-177d-465c-8b8f-60daebc95bc6\") " pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.836856 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.836844 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk97m\" (UniqueName: \"kubernetes.io/projected/475e4f82-6ecd-47d9-88e1-6901ff54ba92-kube-api-access-hk97m\") pod \"dns-default-znj5q\" (UID: \"475e4f82-6ecd-47d9-88e1-6901ff54ba92\") " pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:14.915767 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.915738 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wp7kd" Jul 10 14:46:14.925859 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.925841 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hhcdc" Jul 10 14:46:14.945073 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:14.945047 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:15.081264 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:15.081068 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wp7kd"] Jul 10 14:46:15.086066 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:15.085900 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c6f071_e790_4ccf_909f_131a20bff3ec.slice/crio-8fe3e8f6f1158e10b139536a3416428006ba6402b57ae9f2dc03118b2737d521 WatchSource:0}: Error finding container 8fe3e8f6f1158e10b139536a3416428006ba6402b57ae9f2dc03118b2737d521: Status 404 returned error can't find the container with id 8fe3e8f6f1158e10b139536a3416428006ba6402b57ae9f2dc03118b2737d521 Jul 10 14:46:15.089294 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:15.089251 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hhcdc"] Jul 10 14:46:15.092080 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:15.092058 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd9bc4d0_177d_465c_8b8f_60daebc95bc6.slice/crio-591317d59a531d913735ee34d527eefd1c9ec89ac7a13ac0a3e7641ea5476575 WatchSource:0}: Error finding container 591317d59a531d913735ee34d527eefd1c9ec89ac7a13ac0a3e7641ea5476575: Status 404 returned error can't find the container with id 591317d59a531d913735ee34d527eefd1c9ec89ac7a13ac0a3e7641ea5476575 Jul 10 14:46:15.093797 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:15.093778 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-znj5q"] Jul 10 14:46:15.096655 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:15.096633 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod475e4f82_6ecd_47d9_88e1_6901ff54ba92.slice/crio-d6f46e980b320d4dcd814e6b6a66a27a93bd1c07af5f3debc14aedb4cdcdf4a7 WatchSource:0}: Error finding container d6f46e980b320d4dcd814e6b6a66a27a93bd1c07af5f3debc14aedb4cdcdf4a7: Status 404 returned error can't find the container with id d6f46e980b320d4dcd814e6b6a66a27a93bd1c07af5f3debc14aedb4cdcdf4a7 Jul 10 14:46:15.512017 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:15.511974 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-znj5q" event={"ID":"475e4f82-6ecd-47d9-88e1-6901ff54ba92","Type":"ContainerStarted","Data":"d6f46e980b320d4dcd814e6b6a66a27a93bd1c07af5f3debc14aedb4cdcdf4a7"} Jul 10 14:46:15.512983 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:15.512962 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hhcdc" event={"ID":"fd9bc4d0-177d-465c-8b8f-60daebc95bc6","Type":"ContainerStarted","Data":"591317d59a531d913735ee34d527eefd1c9ec89ac7a13ac0a3e7641ea5476575"} Jul 10 14:46:15.513884 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:15.513863 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wp7kd" event={"ID":"e5c6f071-e790-4ccf-909f-131a20bff3ec","Type":"ContainerStarted","Data":"8fe3e8f6f1158e10b139536a3416428006ba6402b57ae9f2dc03118b2737d521"} Jul 10 14:46:16.274498 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:16.274466 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:16.275273 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:16.275237 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:16.275467 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:16.275447 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:16.278323 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:16.278297 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-55wnk\"" Jul 10 14:46:16.278441 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:16.278421 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 14:46:16.278490 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:16.278464 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 14:46:16.278586 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:16.278569 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 14:46:16.278690 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:16.278674 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 14:46:16.278793 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:16.278778 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gfp6g\"" Jul 10 14:46:18.759075 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.759038 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jmpjq"] Jul 10 14:46:18.824807 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.824775 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jmpjq"] Jul 10 14:46:18.825005 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.824872 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.827492 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.827468 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 10 14:46:18.827686 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.827473 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wxmsn\"" Jul 10 14:46:18.827806 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.827772 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 10 14:46:18.828293 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.828271 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 14:46:18.828293 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.828275 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 14:46:18.828782 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.828761 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 14:46:18.865746 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.863232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.865746 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.863354 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjvjs\" (UniqueName: \"kubernetes.io/projected/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-kube-api-access-wjvjs\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.865746 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.863443 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.865746 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.863473 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.964109 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.964060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.964290 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.964134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjvjs\" (UniqueName: \"kubernetes.io/projected/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-kube-api-access-wjvjs\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.964290 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.964200 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.964290 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.964227 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.965038 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.965012 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.967675 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.967648 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.967795 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.967718 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:18.972997 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:18.972976 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjvjs\" (UniqueName: \"kubernetes.io/projected/fd0267ce-450e-4a55-b7fa-362e7d8dfc2e-kube-api-access-wjvjs\") pod \"prometheus-operator-64785b6675-jmpjq\" (UID: \"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:19.135306 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:19.135218 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" Jul 10 14:46:20.915605 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:20.915573 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jmpjq"] Jul 10 14:46:20.924105 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:20.923977 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0267ce_450e_4a55_b7fa_362e7d8dfc2e.slice/crio-56d05ce178d1a3a4fa75481442bca9a4e020da5b9d7427689c693d0ef9f17ee9 WatchSource:0}: Error finding container 56d05ce178d1a3a4fa75481442bca9a4e020da5b9d7427689c693d0ef9f17ee9: Status 404 returned error can't find the container with id 56d05ce178d1a3a4fa75481442bca9a4e020da5b9d7427689c693d0ef9f17ee9 Jul 10 14:46:21.541879 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:21.541839 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hhcdc" event={"ID":"fd9bc4d0-177d-465c-8b8f-60daebc95bc6","Type":"ContainerStarted","Data":"9e37cd5fe4a161bfabb97b7ddce0abfeb018fda5919ffd640feb7d4dda80e1ec"} Jul 10 14:46:21.542053 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:21.541928 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hhcdc" event={"ID":"fd9bc4d0-177d-465c-8b8f-60daebc95bc6","Type":"ContainerStarted","Data":"698426518cb4d6cc73ff6949c9a5a11519ef10b556f990b5f32e2e11e3ff11d9"} Jul 10 14:46:21.542935 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:21.542907 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" event={"ID":"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e","Type":"ContainerStarted","Data":"56d05ce178d1a3a4fa75481442bca9a4e020da5b9d7427689c693d0ef9f17ee9"} Jul 10 14:46:21.544209 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:21.544190 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wp7kd" event={"ID":"e5c6f071-e790-4ccf-909f-131a20bff3ec","Type":"ContainerStarted","Data":"8d02e60a83764e85461345b13380603fe70943dd6db514b1265c5a96f2c968db"} Jul 10 14:46:21.545759 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:21.545739 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-znj5q" event={"ID":"475e4f82-6ecd-47d9-88e1-6901ff54ba92","Type":"ContainerStarted","Data":"c7a2f9d5403a7fc127ff1003c9b62509cfaed074c5ee4b900c943df15f499c90"} Jul 10 14:46:21.545849 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:21.545775 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-znj5q" event={"ID":"475e4f82-6ecd-47d9-88e1-6901ff54ba92","Type":"ContainerStarted","Data":"1fcfedd346b3b1d2e65d6ba7b6c8e193752b2b304a59589076d18978a11e9e25"} Jul 10 14:46:21.545886 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:21.545868 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:21.561606 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:21.561569 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wp7kd" podStartSLOduration=1.8785145829999998 podStartE2EDuration="7.561557428s" totalImagesPullingTime="5.683042845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:21.561450905 +0000 UTC m=+61.841965603" watchObservedRunningTime="2026-07-10 14:46:21.561557428 +0000 UTC m=+61.842072125" Jul 10 14:46:22.459276 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:22.458056 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zqbmp" Jul 10 14:46:22.491510 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:22.490977 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-znj5q" podStartSLOduration=2.820188096 podStartE2EDuration="8.490962383s" totalImagesPullingTime="5.670774287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:21.58237449 +0000 UTC m=+61.862889187" watchObservedRunningTime="2026-07-10 14:46:22.490962383 +0000 UTC m=+62.771477060" Jul 10 14:46:23.557120 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:23.557075 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" event={"ID":"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e","Type":"ContainerStarted","Data":"76565fd0a4b4f19213e4ee6e42adb39e8d5799d10c0c22a45ecc30d2e7ed6d66"} Jul 10 14:46:23.557594 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:23.557185 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" event={"ID":"fd0267ce-450e-4a55-b7fa-362e7d8dfc2e","Type":"ContainerStarted","Data":"e627bec06e1118d27d61c4e305589928ee6837ba9c58f8e1f8e9dec739fbf492"} Jul 10 14:46:23.576165 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:23.576092 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-jmpjq" podStartSLOduration=3.800726487 podStartE2EDuration="5.576076662s" totalImagesPullingTime="1.775350175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:23.575483881 +0000 UTC m=+63.855998580" watchObservedRunningTime="2026-07-10 14:46:23.576076662 +0000 UTC m=+63.856591360" Jul 10 14:46:24.561819 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:24.561783 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hhcdc" event={"ID":"fd9bc4d0-177d-465c-8b8f-60daebc95bc6","Type":"ContainerStarted","Data":"d7aa09fdfcd6fe11e3998f7e948bcba58eefd223336f5b9fc4baca81c225de50"} Jul 10 14:46:24.580197 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:24.580153 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hhcdc" podStartSLOduration=1.8363199350000001 podStartE2EDuration="10.580138039s" totalImagesPullingTime="8.743818104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:46:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:24.579443275 +0000 UTC m=+64.859957975" watchObservedRunningTime="2026-07-10 14:46:24.580138039 +0000 UTC m=+64.860652730" Jul 10 14:46:25.206850 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.206815 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p"] Jul 10 14:46:25.208993 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.208970 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.210156 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.210129 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7wwz2"] Jul 10 14:46:25.212065 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.212047 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.212647 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.212626 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 10 14:46:25.212773 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.212757 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zgf8w\"" Jul 10 14:46:25.212820 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.212796 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 10 14:46:25.214262 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.214244 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-msnxv\"" Jul 10 14:46:25.214490 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.214466 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 14:46:25.214635 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.214615 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-shfgb"] Jul 10 14:46:25.214780 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.214765 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 14:46:25.216126 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.216042 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 14:46:25.217087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.217070 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.219159 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.219134 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 10 14:46:25.219475 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.219451 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 10 14:46:25.220071 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.220055 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8r8bc\"" Jul 10 14:46:25.220157 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.220100 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 10 14:46:25.234089 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.234067 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p"] Jul 10 14:46:25.269393 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.269359 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-shfgb"] Jul 10 14:46:25.318069 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.318069 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zhqh\" (UniqueName: \"kubernetes.io/projected/c2335d6c-8954-4974-978e-3a28da1d1610-kube-api-access-2zhqh\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.318319 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318114 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-tls\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.318319 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318145 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef251af1-dc02-4555-9c6f-1f3b08fd2174-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.318319 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318172 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2335d6c-8954-4974-978e-3a28da1d1610-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.318319 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318198 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5adbec70-77f2-46b8-bbd1-d28b1f08b280-metrics-client-ca\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.318319 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318221 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.318319 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318244 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5adbec70-77f2-46b8-bbd1-d28b1f08b280-sys\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.318319 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318284 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjj5n\" (UniqueName: \"kubernetes.io/projected/ef251af1-dc02-4555-9c6f-1f3b08fd2174-kube-api-access-hjj5n\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.318608 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5adbec70-77f2-46b8-bbd1-d28b1f08b280-root\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.318608 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318447 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.318608 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318474 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef251af1-dc02-4555-9c6f-1f3b08fd2174-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.318608 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318491 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-wtmp\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.318608 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318533 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdd7w\" (UniqueName: \"kubernetes.io/projected/5adbec70-77f2-46b8-bbd1-d28b1f08b280-kube-api-access-zdd7w\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.318608 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318553 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-accelerators-collector-config\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.318608 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef251af1-dc02-4555-9c6f-1f3b08fd2174-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.318926 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318679 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c2335d6c-8954-4974-978e-3a28da1d1610-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.318926 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318738 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.318926 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.318757 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-textfile\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.420089 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c2335d6c-8954-4974-978e-3a28da1d1610-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.420298 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420111 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.420298 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420137 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-textfile\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.420298 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420167 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.420298 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420194 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zhqh\" (UniqueName: \"kubernetes.io/projected/c2335d6c-8954-4974-978e-3a28da1d1610-kube-api-access-2zhqh\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.420298 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-tls\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.420298 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420250 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef251af1-dc02-4555-9c6f-1f3b08fd2174-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.420298 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2335d6c-8954-4974-978e-3a28da1d1610-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420310 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5adbec70-77f2-46b8-bbd1-d28b1f08b280-metrics-client-ca\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:25.420317 2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5adbec70-77f2-46b8-bbd1-d28b1f08b280-sys\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:25.420422 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-tls podName:c2335d6c-8954-4974-978e-3a28da1d1610 nodeName:}" failed. No retries permitted until 2026-07-10 14:46:25.920399653 +0000 UTC m=+66.200914335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-shfgb" (UID: "c2335d6c-8954-4974-978e-3a28da1d1610") : secret "kube-state-metrics-tls" not found Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420445 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjj5n\" (UniqueName: \"kubernetes.io/projected/ef251af1-dc02-4555-9c6f-1f3b08fd2174-kube-api-access-hjj5n\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420484 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5adbec70-77f2-46b8-bbd1-d28b1f08b280-sys\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420488 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5adbec70-77f2-46b8-bbd1-d28b1f08b280-root\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420532 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5adbec70-77f2-46b8-bbd1-d28b1f08b280-root\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420561 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-textfile\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420561 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c2335d6c-8954-4974-978e-3a28da1d1610-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef251af1-dc02-4555-9c6f-1f3b08fd2174-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420619 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-wtmp\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.420677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdd7w\" (UniqueName: \"kubernetes.io/projected/5adbec70-77f2-46b8-bbd1-d28b1f08b280-kube-api-access-zdd7w\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.421425 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420693 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-accelerators-collector-config\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.421425 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.420727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef251af1-dc02-4555-9c6f-1f3b08fd2174-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.421425 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.421118 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-wtmp\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.421425 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.421236 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.421590 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.421480 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef251af1-dc02-4555-9c6f-1f3b08fd2174-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.421635 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.421594 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2335d6c-8954-4974-978e-3a28da1d1610-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.421865 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.421840 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-accelerators-collector-config\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.422133 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.422104 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5adbec70-77f2-46b8-bbd1-d28b1f08b280-metrics-client-ca\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.423100 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.423078 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.423557 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.423536 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.423663 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.423596 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5adbec70-77f2-46b8-bbd1-d28b1f08b280-node-exporter-tls\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.424013 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.423989 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef251af1-dc02-4555-9c6f-1f3b08fd2174-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.424124 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.424102 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef251af1-dc02-4555-9c6f-1f3b08fd2174-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.432845 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.432825 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zhqh\" (UniqueName: \"kubernetes.io/projected/c2335d6c-8954-4974-978e-3a28da1d1610-kube-api-access-2zhqh\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.433106 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.433088 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjj5n\" (UniqueName: \"kubernetes.io/projected/ef251af1-dc02-4555-9c6f-1f3b08fd2174-kube-api-access-hjj5n\") pod \"openshift-state-metrics-84bd68b896-djt7p\" (UID: \"ef251af1-dc02-4555-9c6f-1f3b08fd2174\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.441662 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.441636 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdd7w\" (UniqueName: \"kubernetes.io/projected/5adbec70-77f2-46b8-bbd1-d28b1f08b280-kube-api-access-zdd7w\") pod \"node-exporter-7wwz2\" (UID: \"5adbec70-77f2-46b8-bbd1-d28b1f08b280\") " pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.518930 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.518900 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" Jul 10 14:46:25.524695 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.524672 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7wwz2" Jul 10 14:46:25.535146 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:25.535118 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5adbec70_77f2_46b8_bbd1_d28b1f08b280.slice/crio-21ee08669b3e7800db14b1e0acee937564660f14798918323eefad0e46b36a02 WatchSource:0}: Error finding container 21ee08669b3e7800db14b1e0acee937564660f14798918323eefad0e46b36a02: Status 404 returned error can't find the container with id 21ee08669b3e7800db14b1e0acee937564660f14798918323eefad0e46b36a02 Jul 10 14:46:25.567548 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.567181 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wwz2" event={"ID":"5adbec70-77f2-46b8-bbd1-d28b1f08b280","Type":"ContainerStarted","Data":"21ee08669b3e7800db14b1e0acee937564660f14798918323eefad0e46b36a02"} Jul 10 14:46:25.644289 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.644252 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p"] Jul 10 14:46:25.647694 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:25.647668 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef251af1_dc02_4555_9c6f_1f3b08fd2174.slice/crio-d6a80b496c55e360abbc4408a4e671259abb247e3f7c15d3168948870d747924 WatchSource:0}: Error finding container d6a80b496c55e360abbc4408a4e671259abb247e3f7c15d3168948870d747924: Status 404 returned error can't find the container with id d6a80b496c55e360abbc4408a4e671259abb247e3f7c15d3168948870d747924 Jul 10 14:46:25.924842 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.924748 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:25.925267 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.925248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:25.925356 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.925309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.927262 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.927243 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 14:46:25.927561 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.927547 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 14:46:25.927730 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.927712 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2335d6c-8954-4974-978e-3a28da1d1610-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-shfgb\" (UID: \"c2335d6c-8954-4974-978e-3a28da1d1610\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:25.937582 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.937563 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47c939b7-87e0-4cc4-a214-d47ce0fb2ee2-metrics-certs\") pod \"network-metrics-daemon-rr8m4\" (UID: \"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2\") " pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:25.937667 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:25.937646 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/756c9aaf-f794-44f5-98fe-3ba47f92e0dd-original-pull-secret\") pod \"global-pull-secret-syncer-dfm2x\" (UID: \"756c9aaf-f794-44f5-98fe-3ba47f92e0dd\") " pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:26.026109 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.026076 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqgh\" (UniqueName: \"kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh\") pod \"network-check-target-g2z2c\" (UID: \"cb1e9cb5-c361-4628-afd2-b341d97047a3\") " pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:26.028482 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.028465 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 14:46:26.039032 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.039009 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 14:46:26.049463 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.049437 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blqgh\" (UniqueName: \"kubernetes.io/projected/cb1e9cb5-c361-4628-afd2-b341d97047a3-kube-api-access-blqgh\") pod \"network-check-target-g2z2c\" (UID: \"cb1e9cb5-c361-4628-afd2-b341d97047a3\") " pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:26.130110 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.130075 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" Jul 10 14:46:26.198743 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.195813 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-55wnk\"" Jul 10 14:46:26.202489 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.202440 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rr8m4" Jul 10 14:46:26.204280 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.204208 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 14:46:26.206322 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.206292 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gfp6g\"" Jul 10 14:46:26.207740 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.207716 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfm2x" Jul 10 14:46:26.208893 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.208682 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.211478 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.211180 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 14:46:26.211478 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.211236 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 14:46:26.211478 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.211290 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 14:46:26.211478 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.211237 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 14:46:26.211723 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.211562 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 14:46:26.212422 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.211943 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 14:46:26.212422 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.211962 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 14:46:26.212422 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.212000 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 14:46:26.212422 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.212088 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:26.212422 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.212154 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p9c8w\"" Jul 10 14:46:26.212422 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.212287 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 14:46:26.224900 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.224300 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 14:46:26.326374 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.325680 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-shfgb"] Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.330704 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.330752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.330784 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-config-volume\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.330810 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj6bb\" (UniqueName: \"kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-kube-api-access-lj6bb\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.330846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-config-out\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.330882 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.330922 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.330949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.330973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-web-config\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.330995 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.331044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.331073 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.333147 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.331100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.403701 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.403668 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dfm2x"] Jul 10 14:46:26.407065 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:26.407039 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756c9aaf_f794_44f5_98fe_3ba47f92e0dd.slice/crio-5eba0e0c5e83d851a8a2a75e15eb99d6ba79a24db2ad1a451b98ba01c12684f2 WatchSource:0}: Error finding container 5eba0e0c5e83d851a8a2a75e15eb99d6ba79a24db2ad1a451b98ba01c12684f2: Status 404 returned error can't find the container with id 5eba0e0c5e83d851a8a2a75e15eb99d6ba79a24db2ad1a451b98ba01c12684f2 Jul 10 14:46:26.415180 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.415153 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rr8m4"] Jul 10 14:46:26.419442 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:26.419413 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47c939b7_87e0_4cc4_a214_d47ce0fb2ee2.slice/crio-0154ab82a1b239c980ce97669dce472d56ffe036411f474fad1d6da935e3e167 WatchSource:0}: Error finding container 0154ab82a1b239c980ce97669dce472d56ffe036411f474fad1d6da935e3e167: Status 404 returned error can't find the container with id 0154ab82a1b239c980ce97669dce472d56ffe036411f474fad1d6da935e3e167 Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432455 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-web-config\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432574 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432601 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432655 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432681 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-config-volume\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432702 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj6bb\" (UniqueName: \"kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-kube-api-access-lj6bb\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432735 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-config-out\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.433692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.432845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.436091 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:26.434388 2562 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 10 14:46:26.436091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.434451 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g2z2c"] Jul 10 14:46:26.436091 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:46:26.434468 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-main-tls podName:7888c5fc-e91a-4621-8127-60b9447b59df nodeName:}" failed. No retries permitted until 2026-07-10 14:46:26.934444819 +0000 UTC m=+67.214959511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df") : secret "alertmanager-main-tls" not found Jul 10 14:46:26.436091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.435101 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.436091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.435242 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.436091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.435455 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.436091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.436044 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.437438 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.437412 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-web-config\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.438185 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.438141 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-config-out\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.438283 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.438255 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.439087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.439034 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-config-volume\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.439966 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.439876 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.439966 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:26.439916 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb1e9cb5_c361_4628_afd2_b341d97047a3.slice/crio-a36cbf94b2d9e60133a35597398c79e785b60d237889a61524623b934c237433 WatchSource:0}: Error finding container a36cbf94b2d9e60133a35597398c79e785b60d237889a61524623b934c237433: Status 404 returned error can't find the container with id a36cbf94b2d9e60133a35597398c79e785b60d237889a61524623b934c237433 Jul 10 14:46:26.440066 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.440038 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.441672 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.441649 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.446029 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.445995 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj6bb\" (UniqueName: \"kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-kube-api-access-lj6bb\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.573139 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.573042 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" event={"ID":"c2335d6c-8954-4974-978e-3a28da1d1610","Type":"ContainerStarted","Data":"0c0860697164a0668865976bd654db77a4d369c358bc421c659f1bf2eaf12ed1"} Jul 10 14:46:26.574817 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.574783 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g2z2c" event={"ID":"cb1e9cb5-c361-4628-afd2-b341d97047a3","Type":"ContainerStarted","Data":"a36cbf94b2d9e60133a35597398c79e785b60d237889a61524623b934c237433"} Jul 10 14:46:26.576625 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.576525 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rr8m4" event={"ID":"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2","Type":"ContainerStarted","Data":"0154ab82a1b239c980ce97669dce472d56ffe036411f474fad1d6da935e3e167"} Jul 10 14:46:26.578025 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.577990 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dfm2x" event={"ID":"756c9aaf-f794-44f5-98fe-3ba47f92e0dd","Type":"ContainerStarted","Data":"5eba0e0c5e83d851a8a2a75e15eb99d6ba79a24db2ad1a451b98ba01c12684f2"} Jul 10 14:46:26.580272 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.580177 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" event={"ID":"ef251af1-dc02-4555-9c6f-1f3b08fd2174","Type":"ContainerStarted","Data":"11797e7d25cc6e91feabe5901944eb3c640ae72c6d423f2f6e76c10ad3d032bb"} Jul 10 14:46:26.580272 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.580230 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" event={"ID":"ef251af1-dc02-4555-9c6f-1f3b08fd2174","Type":"ContainerStarted","Data":"8115097f7bfa7bc338ca0d3aa3e34b849f1a948223b1589a6cca6b8415ca35c5"} Jul 10 14:46:26.580272 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.580250 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" event={"ID":"ef251af1-dc02-4555-9c6f-1f3b08fd2174","Type":"ContainerStarted","Data":"d6a80b496c55e360abbc4408a4e671259abb247e3f7c15d3168948870d747924"} Jul 10 14:46:26.935824 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.935792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:26.938482 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:26.938463 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:27.123084 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:27.123049 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:46:27.311947 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:27.311840 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 14:46:27.318385 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:27.318306 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7888c5fc_e91a_4621_8127_60b9447b59df.slice/crio-c8347ef1e8e04f4888c8e8832a976540dfef5d96ad26ef9655a8cb9f89cf33c3 WatchSource:0}: Error finding container c8347ef1e8e04f4888c8e8832a976540dfef5d96ad26ef9655a8cb9f89cf33c3: Status 404 returned error can't find the container with id c8347ef1e8e04f4888c8e8832a976540dfef5d96ad26ef9655a8cb9f89cf33c3 Jul 10 14:46:27.588210 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:27.588054 2562 generic.go:356] "Generic (PLEG): container finished" podID="5adbec70-77f2-46b8-bbd1-d28b1f08b280" containerID="9ca1a1c41d5f2815930abc274002e1fdd9d983151ddd6e4c305761ac7babb012" exitCode=0 Jul 10 14:46:27.588210 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:27.588147 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wwz2" event={"ID":"5adbec70-77f2-46b8-bbd1-d28b1f08b280","Type":"ContainerDied","Data":"9ca1a1c41d5f2815930abc274002e1fdd9d983151ddd6e4c305761ac7babb012"} Jul 10 14:46:27.591387 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:27.591284 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" event={"ID":"ef251af1-dc02-4555-9c6f-1f3b08fd2174","Type":"ContainerStarted","Data":"b32f588f93e2324fc5342650194296fe15be6e9728c99cfffad4cc6af81b9d67"} Jul 10 14:46:27.594382 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:27.594280 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerStarted","Data":"c8347ef1e8e04f4888c8e8832a976540dfef5d96ad26ef9655a8cb9f89cf33c3"} Jul 10 14:46:28.604777 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:28.604567 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rr8m4" event={"ID":"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2","Type":"ContainerStarted","Data":"0a2587fac4a1cbb3f8dd71be56479c63511f0526cdff2046c62178b7bd35bc2a"} Jul 10 14:46:28.604777 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:28.604642 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rr8m4" event={"ID":"47c939b7-87e0-4cc4-a214-d47ce0fb2ee2","Type":"ContainerStarted","Data":"94ada034b8b9671163e5e79e4a6cbe474448807c02131487c1611a9c6ad89634"} Jul 10 14:46:28.612513 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:28.612482 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wwz2" event={"ID":"5adbec70-77f2-46b8-bbd1-d28b1f08b280","Type":"ContainerStarted","Data":"9b438ec905d0a004432acb9861a19d0b557c30ce0e70617ecbbc9025332905a1"} Jul 10 14:46:28.612682 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:28.612541 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wwz2" event={"ID":"5adbec70-77f2-46b8-bbd1-d28b1f08b280","Type":"ContainerStarted","Data":"8a866af0de4aafc2ec092a9994f49302039a9b626c20e20214b407a65eaa11b2"} Jul 10 14:46:28.617637 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:28.617604 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" event={"ID":"c2335d6c-8954-4974-978e-3a28da1d1610","Type":"ContainerStarted","Data":"12a4740f0ee25a3c598367a229d3434689c19695d5ec72782bddad6e356a5d51"} Jul 10 14:46:28.617757 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:28.617670 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" event={"ID":"c2335d6c-8954-4974-978e-3a28da1d1610","Type":"ContainerStarted","Data":"9f98e382bfcfe6f8af0105c8c7c7b80f399b08ab3cd61a80a894d860738dbee2"} Jul 10 14:46:28.617757 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:28.617702 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" event={"ID":"c2335d6c-8954-4974-978e-3a28da1d1610","Type":"ContainerStarted","Data":"b382a3fcc5399538a68590a9e1469199b14f2f6444bd5732314168a2884784e7"} Jul 10 14:46:28.622516 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:28.622457 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-djt7p" podStartSLOduration=2.451874244 podStartE2EDuration="3.622438781s" totalImagesPullingTime="1.170564537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:27.633829687 +0000 UTC m=+67.914344390" watchObservedRunningTime="2026-07-10 14:46:28.622438781 +0000 UTC m=+68.902953476" Jul 10 14:46:28.624015 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:28.623698 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rr8m4" podStartSLOduration=65.832195721 podStartE2EDuration="1m7.623685748s" totalImagesPullingTime="1.791490027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:28.621676541 +0000 UTC m=+68.902191239" watchObservedRunningTime="2026-07-10 14:46:28.623685748 +0000 UTC m=+68.904200447" Jul 10 14:46:28.643488 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:28.643120 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-shfgb" podStartSLOduration=1.768167177 podStartE2EDuration="3.643100997s" totalImagesPullingTime="1.87493382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:28.642214046 +0000 UTC m=+68.922728745" watchObservedRunningTime="2026-07-10 14:46:28.643100997 +0000 UTC m=+68.923615696" Jul 10 14:46:28.667242 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:28.666980 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7wwz2" podStartSLOduration=2.17273473 podStartE2EDuration="3.66696084s" totalImagesPullingTime="1.396752031s" totalInitContainerRuntime="97.474079ms" isStatefulPod=true podCreationTimestamp="2026-07-10 14:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:28.665808577 +0000 UTC m=+68.946323289" watchObservedRunningTime="2026-07-10 14:46:28.66696084 +0000 UTC m=+68.947475539" Jul 10 14:46:29.631263 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.631232 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg"] Jul 10 14:46:29.639590 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.638898 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.643969 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.643788 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 10 14:46:29.644917 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.644719 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 10 14:46:29.644917 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.644774 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 14:46:29.644917 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.644808 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-frmb365f28b6k\"" Jul 10 14:46:29.644917 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.644833 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-j2zvx\"" Jul 10 14:46:29.644917 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.644729 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 10 14:46:29.649747 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.649726 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg"] Jul 10 14:46:29.762299 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.762256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17cacce8-2719-4477-9f5f-16d924a2b1f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.762299 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.762303 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/17cacce8-2719-4477-9f5f-16d924a2b1f3-secret-metrics-server-tls\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.762582 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.762390 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/17cacce8-2719-4477-9f5f-16d924a2b1f3-audit-log\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.762582 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.762456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/17cacce8-2719-4477-9f5f-16d924a2b1f3-client-ca-bundle\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.762582 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.762500 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/17cacce8-2719-4477-9f5f-16d924a2b1f3-metrics-server-audit-profiles\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.762582 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.762530 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdm7z\" (UniqueName: \"kubernetes.io/projected/17cacce8-2719-4477-9f5f-16d924a2b1f3-kube-api-access-gdm7z\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.762582 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.762569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/17cacce8-2719-4477-9f5f-16d924a2b1f3-secret-metrics-server-client-certs\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.863662 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.863627 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/17cacce8-2719-4477-9f5f-16d924a2b1f3-client-ca-bundle\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.863817 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.863668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/17cacce8-2719-4477-9f5f-16d924a2b1f3-metrics-server-audit-profiles\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.863817 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.863695 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdm7z\" (UniqueName: \"kubernetes.io/projected/17cacce8-2719-4477-9f5f-16d924a2b1f3-kube-api-access-gdm7z\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.863938 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.863875 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/17cacce8-2719-4477-9f5f-16d924a2b1f3-secret-metrics-server-client-certs\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.863984 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.863938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17cacce8-2719-4477-9f5f-16d924a2b1f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.864039 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.863976 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/17cacce8-2719-4477-9f5f-16d924a2b1f3-secret-metrics-server-tls\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.864099 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.864038 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/17cacce8-2719-4477-9f5f-16d924a2b1f3-audit-log\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.864544 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.864495 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/17cacce8-2719-4477-9f5f-16d924a2b1f3-audit-log\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.864855 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.864799 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/17cacce8-2719-4477-9f5f-16d924a2b1f3-metrics-server-audit-profiles\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.864933 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.864860 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17cacce8-2719-4477-9f5f-16d924a2b1f3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.868367 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.868348 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/17cacce8-2719-4477-9f5f-16d924a2b1f3-secret-metrics-server-client-certs\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.869360 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.869220 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/17cacce8-2719-4477-9f5f-16d924a2b1f3-client-ca-bundle\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.869606 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.869590 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/17cacce8-2719-4477-9f5f-16d924a2b1f3-secret-metrics-server-tls\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.878469 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.878444 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdm7z\" (UniqueName: \"kubernetes.io/projected/17cacce8-2719-4477-9f5f-16d924a2b1f3-kube-api-access-gdm7z\") pod \"metrics-server-5bdbb7ccff-ns9qg\" (UID: \"17cacce8-2719-4477-9f5f-16d924a2b1f3\") " pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:29.950987 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:29.950738 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:31.553529 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:31.553496 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-znj5q" Jul 10 14:46:32.309722 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.309692 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg"] Jul 10 14:46:32.347227 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:32.347190 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17cacce8_2719_4477_9f5f_16d924a2b1f3.slice/crio-5aa4b4e8d37b50798146c6976f0a5436e7680bd0ebc1449129335b6024ec37bf WatchSource:0}: Error finding container 5aa4b4e8d37b50798146c6976f0a5436e7680bd0ebc1449129335b6024ec37bf: Status 404 returned error can't find the container with id 5aa4b4e8d37b50798146c6976f0a5436e7680bd0ebc1449129335b6024ec37bf Jul 10 14:46:32.484549 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.484465 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-gmsr4"] Jul 10 14:46:32.486667 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.486644 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gmsr4" Jul 10 14:46:32.491234 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.491206 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-5wfbc\"" Jul 10 14:46:32.491581 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.491566 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 14:46:32.491672 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.491639 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 14:46:32.513211 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.513178 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gmsr4"] Jul 10 14:46:32.591621 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.591580 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqqb7\" (UniqueName: \"kubernetes.io/projected/1d635b58-4dac-46d2-8398-e652e623cf6c-kube-api-access-mqqb7\") pod \"downloads-68c9cbb497-gmsr4\" (UID: \"1d635b58-4dac-46d2-8398-e652e623cf6c\") " pod="openshift-console/downloads-68c9cbb497-gmsr4" Jul 10 14:46:32.635353 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.635299 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g2z2c" event={"ID":"cb1e9cb5-c361-4628-afd2-b341d97047a3","Type":"ContainerStarted","Data":"5e0f38e3d5ab37a73b5397724a4db93655445c12b1674ec35fdf58458f96b84d"} Jul 10 14:46:32.635525 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.635455 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:46:32.636626 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.636595 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" event={"ID":"17cacce8-2719-4477-9f5f-16d924a2b1f3","Type":"ContainerStarted","Data":"5aa4b4e8d37b50798146c6976f0a5436e7680bd0ebc1449129335b6024ec37bf"} Jul 10 14:46:32.638043 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.638023 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dfm2x" event={"ID":"756c9aaf-f794-44f5-98fe-3ba47f92e0dd","Type":"ContainerStarted","Data":"923dd69be08b5c7ac42cd57448135825dba97f146bc73e1dba8bc5a55d0ad6f2"} Jul 10 14:46:32.639419 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.639399 2562 generic.go:356] "Generic (PLEG): container finished" podID="7888c5fc-e91a-4621-8127-60b9447b59df" containerID="57d95a4a63143e0c2285ba0509b1c602190a13a3d4e8911a40d422f5a01329a8" exitCode=0 Jul 10 14:46:32.639497 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.639441 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerDied","Data":"57d95a4a63143e0c2285ba0509b1c602190a13a3d4e8911a40d422f5a01329a8"} Jul 10 14:46:32.653120 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.653063 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g2z2c" podStartSLOduration=65.933716092 podStartE2EDuration="1m11.653045509s" totalImagesPullingTime="5.719329417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:32.6524387 +0000 UTC m=+72.932953398" watchObservedRunningTime="2026-07-10 14:46:32.653045509 +0000 UTC m=+72.933560207" Jul 10 14:46:32.692014 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.691980 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqqb7\" (UniqueName: \"kubernetes.io/projected/1d635b58-4dac-46d2-8398-e652e623cf6c-kube-api-access-mqqb7\") pod \"downloads-68c9cbb497-gmsr4\" (UID: \"1d635b58-4dac-46d2-8398-e652e623cf6c\") " pod="openshift-console/downloads-68c9cbb497-gmsr4" Jul 10 14:46:32.707523 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.707471 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dfm2x" podStartSLOduration=65.951176155 podStartE2EDuration="1m11.707454603s" totalImagesPullingTime="5.756278448s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:32.672665933 +0000 UTC m=+72.953180630" watchObservedRunningTime="2026-07-10 14:46:32.707454603 +0000 UTC m=+72.987969295" Jul 10 14:46:32.708935 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.708914 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqqb7\" (UniqueName: \"kubernetes.io/projected/1d635b58-4dac-46d2-8398-e652e623cf6c-kube-api-access-mqqb7\") pod \"downloads-68c9cbb497-gmsr4\" (UID: \"1d635b58-4dac-46d2-8398-e652e623cf6c\") " pod="openshift-console/downloads-68c9cbb497-gmsr4" Jul 10 14:46:32.800711 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.800666 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gmsr4" Jul 10 14:46:32.957727 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:32.957697 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gmsr4"] Jul 10 14:46:32.960441 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:32.960414 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d635b58_4dac_46d2_8398_e652e623cf6c.slice/crio-98d0440355e86d0fdee92dcd7b5ded826ee4662928755bde96d21045c3fbb595 WatchSource:0}: Error finding container 98d0440355e86d0fdee92dcd7b5ded826ee4662928755bde96d21045c3fbb595: Status 404 returned error can't find the container with id 98d0440355e86d0fdee92dcd7b5ded826ee4662928755bde96d21045c3fbb595 Jul 10 14:46:33.644561 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:33.644525 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gmsr4" event={"ID":"1d635b58-4dac-46d2-8398-e652e623cf6c","Type":"ContainerStarted","Data":"98d0440355e86d0fdee92dcd7b5ded826ee4662928755bde96d21045c3fbb595"} Jul 10 14:46:34.655478 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:34.655441 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerStarted","Data":"61ca9d40a4e33be94377014296c07aaa1c5ba69688f419832a1d262e13b0d20c"} Jul 10 14:46:34.655894 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:34.655526 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerStarted","Data":"b83f9ca1ed0dda08fc96f3528168da904c7c77ee5bd583f57d5abacc2bbab270"} Jul 10 14:46:34.655894 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:34.655573 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerStarted","Data":"ce8d4921432acd06b79da7aa1ef5bbc91dc6c95a0297993469c977f5d54cf63e"} Jul 10 14:46:34.657503 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:34.657454 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" event={"ID":"17cacce8-2719-4477-9f5f-16d924a2b1f3","Type":"ContainerStarted","Data":"3b6424adbe8cf79da6357ad6c4c0facf92770129faa735e2f63874d1b274c9d1"} Jul 10 14:46:34.685888 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:34.685838 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" podStartSLOduration=3.654466461 podStartE2EDuration="5.685819601s" totalImagesPullingTime="2.03135314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:34.684147923 +0000 UTC m=+74.964662621" watchObservedRunningTime="2026-07-10 14:46:34.685819601 +0000 UTC m=+74.966334304" Jul 10 14:46:35.664023 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:35.663986 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerStarted","Data":"1cbf58dff4df9092338b7be800511b3fa7b6f37b0f8245cf6e5a5ec45c390d0c"} Jul 10 14:46:35.664578 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:35.664107 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerStarted","Data":"08dd84d3aff4da2a9c797ba33970254d959a0126ba1295d7418de28bbedb1dc5"} Jul 10 14:46:36.672597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:36.672552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerStarted","Data":"a5b10b44803de767667fb24eb7c40b35ea6c56d4665161b780a5a0db5358b660"} Jul 10 14:46:36.706521 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:36.706464 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.25633747 podStartE2EDuration="10.706443346s" totalImagesPullingTime="7.425842306s" totalInitContainerRuntime="24.26357ms" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:36.706030141 +0000 UTC m=+76.986544839" watchObservedRunningTime="2026-07-10 14:46:36.706443346 +0000 UTC m=+76.986958044" Jul 10 14:46:40.657518 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.657410 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8676559bcf-77n6l"] Jul 10 14:46:40.660596 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.660574 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.666103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.666079 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 14:46:40.666568 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.666550 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 14:46:40.671859 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.671838 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 14:46:40.674305 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.674289 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 14:46:40.675597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.675577 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 14:46:40.676189 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.676173 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l7962\"" Jul 10 14:46:40.760914 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.760880 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-serving-cert\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.760914 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.760911 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q2vc\" (UniqueName: \"kubernetes.io/projected/c9450eca-68a7-4622-9c0a-d34ee8b80000-kube-api-access-4q2vc\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.761111 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.760960 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-config\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.761111 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.760981 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-service-ca\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.761111 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.760999 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-oauth-config\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.761111 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.761013 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-oauth-serving-cert\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.776031 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.776006 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8676559bcf-77n6l"] Jul 10 14:46:40.861846 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.861815 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-serving-cert\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.861846 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.861849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q2vc\" (UniqueName: \"kubernetes.io/projected/c9450eca-68a7-4622-9c0a-d34ee8b80000-kube-api-access-4q2vc\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.862110 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.862010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-config\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.862110 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.862046 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-service-ca\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.862224 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.862206 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-oauth-config\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.862260 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.862243 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-oauth-serving-cert\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.862770 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.862748 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-config\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.862881 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.862836 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-service-ca\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.862881 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.862847 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-oauth-serving-cert\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.864721 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.864696 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-serving-cert\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.865217 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.865189 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-oauth-config\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.875063 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.875034 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q2vc\" (UniqueName: \"kubernetes.io/projected/c9450eca-68a7-4622-9c0a-d34ee8b80000-kube-api-access-4q2vc\") pod \"console-8676559bcf-77n6l\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:40.972161 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:40.972076 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:41.123727 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:41.123469 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8676559bcf-77n6l"] Jul 10 14:46:41.130045 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:41.130016 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9450eca_68a7_4622_9c0a_d34ee8b80000.slice/crio-bb5dd5888f0a0d21280d8b99ad76f9ad866f78502e5b733a1bb59ef343e468df WatchSource:0}: Error finding container bb5dd5888f0a0d21280d8b99ad76f9ad866f78502e5b733a1bb59ef343e468df: Status 404 returned error can't find the container with id bb5dd5888f0a0d21280d8b99ad76f9ad866f78502e5b733a1bb59ef343e468df Jul 10 14:46:41.693256 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:41.693221 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8676559bcf-77n6l" event={"ID":"c9450eca-68a7-4622-9c0a-d34ee8b80000","Type":"ContainerStarted","Data":"bb5dd5888f0a0d21280d8b99ad76f9ad866f78502e5b733a1bb59ef343e468df"} Jul 10 14:46:44.709650 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:44.709554 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8676559bcf-77n6l" event={"ID":"c9450eca-68a7-4622-9c0a-d34ee8b80000","Type":"ContainerStarted","Data":"9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d"} Jul 10 14:46:49.951968 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:49.951928 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:49.952446 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:49.952032 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:46:50.214458 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.214324 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8676559bcf-77n6l" podStartSLOduration=6.931603297 podStartE2EDuration="10.214305046s" totalImagesPullingTime="3.282701749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:44.736582716 +0000 UTC m=+85.017097414" watchObservedRunningTime="2026-07-10 14:46:50.214305046 +0000 UTC m=+90.494819748" Jul 10 14:46:50.215286 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.215265 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-578f6574db-fq7f8"] Jul 10 14:46:50.286954 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.286921 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.292906 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.292883 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-578f6574db-fq7f8"] Jul 10 14:46:50.295262 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.295231 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 14:46:50.447990 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.447955 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-oauth-config\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.447990 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.447999 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-trusted-ca-bundle\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.448310 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.448031 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-service-ca\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.448310 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.448092 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-oauth-serving-cert\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.448310 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.448141 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5ltl\" (UniqueName: \"kubernetes.io/projected/11660297-7d14-42a4-8829-28dfd5d8ecc8-kube-api-access-l5ltl\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.448310 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.448165 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-config\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.448310 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.448186 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-serving-cert\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.548724 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.548688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-oauth-config\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.548910 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.548741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-trusted-ca-bundle\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.548910 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.548774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-service-ca\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.548910 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.548798 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-oauth-serving-cert\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.548910 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.548819 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5ltl\" (UniqueName: \"kubernetes.io/projected/11660297-7d14-42a4-8829-28dfd5d8ecc8-kube-api-access-l5ltl\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.548910 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.548844 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-config\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.548910 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.548870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-serving-cert\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.549606 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.549581 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-service-ca\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.549745 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.549643 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-oauth-serving-cert\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.549859 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.549838 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-config\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.550063 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.550043 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-trusted-ca-bundle\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.551970 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.551946 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-oauth-config\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.552183 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.552155 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-serving-cert\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.559108 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.559083 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5ltl\" (UniqueName: \"kubernetes.io/projected/11660297-7d14-42a4-8829-28dfd5d8ecc8-kube-api-access-l5ltl\") pod \"console-578f6574db-fq7f8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.599102 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.599064 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:46:50.972988 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.972904 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:50.973528 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.972986 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:50.978360 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:50.978310 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:51.744017 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:51.743989 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:46:54.254347 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:54.254296 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-578f6574db-fq7f8"] Jul 10 14:46:54.257658 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:46:54.257626 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11660297_7d14_42a4_8829_28dfd5d8ecc8.slice/crio-a65047be4cc89b553d451473c76ffb6cbd9c5b1d5fc6f54e22c1d83dbf67407b WatchSource:0}: Error finding container a65047be4cc89b553d451473c76ffb6cbd9c5b1d5fc6f54e22c1d83dbf67407b: Status 404 returned error can't find the container with id a65047be4cc89b553d451473c76ffb6cbd9c5b1d5fc6f54e22c1d83dbf67407b Jul 10 14:46:54.753536 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:54.753493 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gmsr4" event={"ID":"1d635b58-4dac-46d2-8398-e652e623cf6c","Type":"ContainerStarted","Data":"958bb2928615f3c7fdd1ef2ccd61e65414e1080e8a89c57497f1b6228e15c79c"} Jul 10 14:46:54.753722 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:54.753692 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-gmsr4" Jul 10 14:46:54.755041 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:54.755017 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-578f6574db-fq7f8" event={"ID":"11660297-7d14-42a4-8829-28dfd5d8ecc8","Type":"ContainerStarted","Data":"d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c"} Jul 10 14:46:54.755161 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:54.755056 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-578f6574db-fq7f8" event={"ID":"11660297-7d14-42a4-8829-28dfd5d8ecc8","Type":"ContainerStarted","Data":"a65047be4cc89b553d451473c76ffb6cbd9c5b1d5fc6f54e22c1d83dbf67407b"} Jul 10 14:46:54.755261 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:54.755240 2562 patch_prober.go:28] interesting pod/downloads-68c9cbb497-gmsr4 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.14:8080/\": dial tcp 10.134.0.14:8080: connect: connection refused" start-of-body= Jul 10 14:46:54.755323 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:54.755292 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-gmsr4" podUID="1d635b58-4dac-46d2-8398-e652e623cf6c" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.14:8080/\": dial tcp 10.134.0.14:8080: connect: connection refused" Jul 10 14:46:54.776371 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:54.776284 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-gmsr4" podStartSLOduration=1.178610707 podStartE2EDuration="22.776268395s" totalImagesPullingTime="21.597657688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:54.775145651 +0000 UTC m=+95.055660350" watchObservedRunningTime="2026-07-10 14:46:54.776268395 +0000 UTC m=+95.056783092" Jul 10 14:46:54.796979 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:54.796929 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-578f6574db-fq7f8" podStartSLOduration=4.796914185 podStartE2EDuration="4.796914185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:54.795883991 +0000 UTC m=+95.076398690" watchObservedRunningTime="2026-07-10 14:46:54.796914185 +0000 UTC m=+95.077428885" Jul 10 14:46:55.776801 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:46:55.776763 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-gmsr4" Jul 10 14:47:00.600175 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:00.600134 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:47:00.600920 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:00.600894 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:47:00.606052 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:00.606029 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:47:00.792780 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:00.792745 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:47:00.848806 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:00.848772 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8676559bcf-77n6l"] Jul 10 14:47:03.647291 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:03.646861 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g2z2c" Jul 10 14:47:09.956637 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:09.956608 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:47:09.960408 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:09.960383 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5bdbb7ccff-ns9qg" Jul 10 14:47:25.879507 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:25.879429 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8676559bcf-77n6l" podUID="c9450eca-68a7-4622-9c0a-d34ee8b80000" containerName="console" containerID="cri-o://9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d" gracePeriod=15 Jul 10 14:47:26.162193 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.162127 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:47:26.266366 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.266317 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-oauth-config\") pod \"c9450eca-68a7-4622-9c0a-d34ee8b80000\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " Jul 10 14:47:26.266366 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.266366 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-serving-cert\") pod \"c9450eca-68a7-4622-9c0a-d34ee8b80000\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " Jul 10 14:47:26.266602 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.266406 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q2vc\" (UniqueName: \"kubernetes.io/projected/c9450eca-68a7-4622-9c0a-d34ee8b80000-kube-api-access-4q2vc\") pod \"c9450eca-68a7-4622-9c0a-d34ee8b80000\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " Jul 10 14:47:26.266602 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.266429 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-config\") pod \"c9450eca-68a7-4622-9c0a-d34ee8b80000\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " Jul 10 14:47:26.266602 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.266450 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-service-ca\") pod \"c9450eca-68a7-4622-9c0a-d34ee8b80000\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " Jul 10 14:47:26.266602 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.266489 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-oauth-serving-cert\") pod \"c9450eca-68a7-4622-9c0a-d34ee8b80000\" (UID: \"c9450eca-68a7-4622-9c0a-d34ee8b80000\") " Jul 10 14:47:26.266851 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.266817 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-config" (OuterVolumeSpecName: "console-config") pod "c9450eca-68a7-4622-9c0a-d34ee8b80000" (UID: "c9450eca-68a7-4622-9c0a-d34ee8b80000"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 14:47:26.266954 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.266869 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-service-ca" (OuterVolumeSpecName: "service-ca") pod "c9450eca-68a7-4622-9c0a-d34ee8b80000" (UID: "c9450eca-68a7-4622-9c0a-d34ee8b80000"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 14:47:26.266954 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.266904 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c9450eca-68a7-4622-9c0a-d34ee8b80000" (UID: "c9450eca-68a7-4622-9c0a-d34ee8b80000"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 14:47:26.268804 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.268770 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c9450eca-68a7-4622-9c0a-d34ee8b80000" (UID: "c9450eca-68a7-4622-9c0a-d34ee8b80000"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:47:26.268903 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.268843 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c9450eca-68a7-4622-9c0a-d34ee8b80000" (UID: "c9450eca-68a7-4622-9c0a-d34ee8b80000"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:47:26.268956 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.268895 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9450eca-68a7-4622-9c0a-d34ee8b80000-kube-api-access-4q2vc" (OuterVolumeSpecName: "kube-api-access-4q2vc") pod "c9450eca-68a7-4622-9c0a-d34ee8b80000" (UID: "c9450eca-68a7-4622-9c0a-d34ee8b80000"). InnerVolumeSpecName "kube-api-access-4q2vc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:47:26.368050 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.368013 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-oauth-serving-cert\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:26.368050 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.368043 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-oauth-config\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:26.368050 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.368052 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-serving-cert\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:26.368050 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.368060 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q2vc\" (UniqueName: \"kubernetes.io/projected/c9450eca-68a7-4622-9c0a-d34ee8b80000-kube-api-access-4q2vc\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:26.368316 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.368070 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-console-config\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:26.368316 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.368080 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9450eca-68a7-4622-9c0a-d34ee8b80000-service-ca\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:26.888566 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.888535 2562 generic.go:356] "Generic (PLEG): container finished" podID="c9450eca-68a7-4622-9c0a-d34ee8b80000" containerID="9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d" exitCode=0 Jul 10 14:47:26.889034 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.888620 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8676559bcf-77n6l" event={"ID":"c9450eca-68a7-4622-9c0a-d34ee8b80000","Type":"ContainerDied","Data":"9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d"} Jul 10 14:47:26.889034 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.888654 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8676559bcf-77n6l" Jul 10 14:47:26.889034 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.888671 2562 scope.go:117] "RemoveContainer" containerID="9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d" Jul 10 14:47:26.889034 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.888661 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8676559bcf-77n6l" event={"ID":"c9450eca-68a7-4622-9c0a-d34ee8b80000","Type":"ContainerDied","Data":"bb5dd5888f0a0d21280d8b99ad76f9ad866f78502e5b733a1bb59ef343e468df"} Jul 10 14:47:26.901741 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.901725 2562 scope.go:117] "RemoveContainer" containerID="9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d" Jul 10 14:47:26.901987 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:47:26.901968 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d\": container with ID starting with 9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d not found: ID does not exist" containerID="9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d" Jul 10 14:47:26.902033 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.901997 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d"} err="failed to get container status \"9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d\": rpc error: code = NotFound desc = could not find container \"9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d\": container with ID starting with 9253a5dc708464ec2511c29c7e020f1df0463e03d522ce56f8c378b2ecc3709d not found: ID does not exist" Jul 10 14:47:26.905690 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.905669 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8676559bcf-77n6l"] Jul 10 14:47:26.909930 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:26.909912 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8676559bcf-77n6l"] Jul 10 14:47:28.278581 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:28.278550 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9450eca-68a7-4622-9c0a-d34ee8b80000" path="/var/lib/kubelet/pods/c9450eca-68a7-4622-9c0a-d34ee8b80000/volumes" Jul 10 14:47:45.733961 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.733929 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 14:47:45.734463 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.734212 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="alertmanager" containerID="cri-o://ce8d4921432acd06b79da7aa1ef5bbc91dc6c95a0297993469c977f5d54cf63e" gracePeriod=120 Jul 10 14:47:45.734463 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.734273 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy-web" containerID="cri-o://61ca9d40a4e33be94377014296c07aaa1c5ba69688f419832a1d262e13b0d20c" gracePeriod=120 Jul 10 14:47:45.734463 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.734265 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy-metric" containerID="cri-o://1cbf58dff4df9092338b7be800511b3fa7b6f37b0f8245cf6e5a5ec45c390d0c" gracePeriod=120 Jul 10 14:47:45.734463 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.734305 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="config-reloader" containerID="cri-o://b83f9ca1ed0dda08fc96f3528168da904c7c77ee5bd583f57d5abacc2bbab270" gracePeriod=120 Jul 10 14:47:45.734463 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.734381 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="prom-label-proxy" containerID="cri-o://a5b10b44803de767667fb24eb7c40b35ea6c56d4665161b780a5a0db5358b660" gracePeriod=120 Jul 10 14:47:45.734463 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.734387 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy" containerID="cri-o://08dd84d3aff4da2a9c797ba33970254d959a0126ba1295d7418de28bbedb1dc5" gracePeriod=120 Jul 10 14:47:45.960070 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.960039 2562 generic.go:356] "Generic (PLEG): container finished" podID="7888c5fc-e91a-4621-8127-60b9447b59df" containerID="a5b10b44803de767667fb24eb7c40b35ea6c56d4665161b780a5a0db5358b660" exitCode=0 Jul 10 14:47:45.960070 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.960063 2562 generic.go:356] "Generic (PLEG): container finished" podID="7888c5fc-e91a-4621-8127-60b9447b59df" containerID="1cbf58dff4df9092338b7be800511b3fa7b6f37b0f8245cf6e5a5ec45c390d0c" exitCode=0 Jul 10 14:47:45.960070 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.960072 2562 generic.go:356] "Generic (PLEG): container finished" podID="7888c5fc-e91a-4621-8127-60b9447b59df" containerID="08dd84d3aff4da2a9c797ba33970254d959a0126ba1295d7418de28bbedb1dc5" exitCode=0 Jul 10 14:47:45.960070 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.960079 2562 generic.go:356] "Generic (PLEG): container finished" podID="7888c5fc-e91a-4621-8127-60b9447b59df" containerID="b83f9ca1ed0dda08fc96f3528168da904c7c77ee5bd583f57d5abacc2bbab270" exitCode=0 Jul 10 14:47:45.960070 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.960090 2562 generic.go:356] "Generic (PLEG): container finished" podID="7888c5fc-e91a-4621-8127-60b9447b59df" containerID="ce8d4921432acd06b79da7aa1ef5bbc91dc6c95a0297993469c977f5d54cf63e" exitCode=0 Jul 10 14:47:45.960381 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.960120 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerDied","Data":"a5b10b44803de767667fb24eb7c40b35ea6c56d4665161b780a5a0db5358b660"} Jul 10 14:47:45.960381 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.960193 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerDied","Data":"1cbf58dff4df9092338b7be800511b3fa7b6f37b0f8245cf6e5a5ec45c390d0c"} Jul 10 14:47:45.960381 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.960221 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerDied","Data":"08dd84d3aff4da2a9c797ba33970254d959a0126ba1295d7418de28bbedb1dc5"} Jul 10 14:47:45.960381 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.960255 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerDied","Data":"b83f9ca1ed0dda08fc96f3528168da904c7c77ee5bd583f57d5abacc2bbab270"} Jul 10 14:47:45.960381 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:45.960278 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerDied","Data":"ce8d4921432acd06b79da7aa1ef5bbc91dc6c95a0297993469c977f5d54cf63e"} Jul 10 14:47:46.967258 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:46.967226 2562 generic.go:356] "Generic (PLEG): container finished" podID="7888c5fc-e91a-4621-8127-60b9447b59df" containerID="61ca9d40a4e33be94377014296c07aaa1c5ba69688f419832a1d262e13b0d20c" exitCode=0 Jul 10 14:47:46.967599 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:46.967265 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerDied","Data":"61ca9d40a4e33be94377014296c07aaa1c5ba69688f419832a1d262e13b0d20c"} Jul 10 14:47:46.967599 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:46.967318 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7888c5fc-e91a-4621-8127-60b9447b59df","Type":"ContainerDied","Data":"c8347ef1e8e04f4888c8e8832a976540dfef5d96ad26ef9655a8cb9f89cf33c3"} Jul 10 14:47:46.967599 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:46.967365 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8347ef1e8e04f4888c8e8832a976540dfef5d96ad26ef9655a8cb9f89cf33c3" Jul 10 14:47:46.968482 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:46.968462 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:47.028798 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.028768 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lj6bb\" (UniqueName: \"kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-kube-api-access-lj6bb\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.028798 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.028805 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-config-volume\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029032 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.028825 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-web-config\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029032 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.028854 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-metrics-client-ca\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029032 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.028971 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-main-tls\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029032 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.029002 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-tls-assets\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029032 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.029030 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-config-out\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029269 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.029054 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-metric\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029269 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.029093 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-web\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029269 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.029119 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029269 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.029156 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-trusted-ca-bundle\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029269 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.029191 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-main-db\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029269 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.029225 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-cluster-tls-config\") pod \"7888c5fc-e91a-4621-8127-60b9447b59df\" (UID: \"7888c5fc-e91a-4621-8127-60b9447b59df\") " Jul 10 14:47:47.029647 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.029530 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 14:47:47.030031 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.030003 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-metrics-client-ca\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.033317 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.033245 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:47:47.033495 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.033258 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 14:47:47.034131 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.034095 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-config-volume" (OuterVolumeSpecName: "config-volume") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:47:47.035326 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.035300 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-config-out" (OuterVolumeSpecName: "config-out") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:47:47.035477 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.035422 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-kube-api-access-lj6bb" (OuterVolumeSpecName: "kube-api-access-lj6bb") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "kube-api-access-lj6bb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:47:47.035542 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.035509 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:47:47.035542 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.035521 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:47:47.035649 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.035568 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:47:47.036129 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.036094 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:47:47.037082 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.037055 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:47:47.039574 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.039548 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:47:47.044843 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.044717 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-web-config" (OuterVolumeSpecName: "web-config") pod "7888c5fc-e91a-4621-8127-60b9447b59df" (UID: "7888c5fc-e91a-4621-8127-60b9447b59df"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:47:47.130938 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.130909 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj6bb\" (UniqueName: \"kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-kube-api-access-lj6bb\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.130938 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.130937 2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-config-volume\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.130938 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.130946 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-web-config\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.131170 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.130955 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-main-tls\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.131170 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.130965 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7888c5fc-e91a-4621-8127-60b9447b59df-tls-assets\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.131170 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.130973 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-config-out\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.131170 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.130981 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.131170 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.130990 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.131170 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.130999 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.131170 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.131009 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.131170 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.131018 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7888c5fc-e91a-4621-8127-60b9447b59df-alertmanager-main-db\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.131170 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.131028 2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7888c5fc-e91a-4621-8127-60b9447b59df-cluster-tls-config\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:47:47.970997 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.970965 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:47.997551 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:47.997494 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 14:47:48.001674 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.001650 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 14:47:48.037870 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.037806 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 14:47:48.038068 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038057 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="config-reloader" Jul 10 14:47:48.038108 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038070 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="config-reloader" Jul 10 14:47:48.038108 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038081 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="alertmanager" Jul 10 14:47:48.038108 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038086 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="alertmanager" Jul 10 14:47:48.038108 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038092 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="prom-label-proxy" Jul 10 14:47:48.038108 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038097 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="prom-label-proxy" Jul 10 14:47:48.038108 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038106 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy" Jul 10 14:47:48.038108 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038110 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy" Jul 10 14:47:48.038308 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038119 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy-metric" Jul 10 14:47:48.038308 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038138 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy-metric" Jul 10 14:47:48.038308 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038147 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy-web" Jul 10 14:47:48.038308 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038152 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy-web" Jul 10 14:47:48.038308 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038160 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9450eca-68a7-4622-9c0a-d34ee8b80000" containerName="console" Jul 10 14:47:48.038308 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038165 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9450eca-68a7-4622-9c0a-d34ee8b80000" containerName="console" Jul 10 14:47:48.038308 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038178 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="init-config-reloader" Jul 10 14:47:48.038308 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038183 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="init-config-reloader" Jul 10 14:47:48.038730 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038707 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="alertmanager" Jul 10 14:47:48.038786 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038734 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy-metric" Jul 10 14:47:48.038786 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038759 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy-web" Jul 10 14:47:48.038786 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038770 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="config-reloader" Jul 10 14:47:48.038786 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038785 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9450eca-68a7-4622-9c0a-d34ee8b80000" containerName="console" Jul 10 14:47:48.038961 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038795 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="prom-label-proxy" Jul 10 14:47:48.038961 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.038818 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" containerName="kube-rbac-proxy" Jul 10 14:47:48.043634 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.043599 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.046460 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.046436 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 14:47:48.046578 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.046436 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 14:47:48.046578 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.046554 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p9c8w\"" Jul 10 14:47:48.047034 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.046880 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 14:47:48.047034 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.046905 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 14:47:48.047034 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.046952 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 14:47:48.047034 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.046992 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 14:47:48.047034 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.046997 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 14:47:48.047436 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.047231 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 14:47:48.054252 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.054234 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 14:47:48.060743 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.060719 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 14:47:48.144604 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144567 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-config-out\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.144604 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144607 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.144854 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144644 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.144854 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144687 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.144854 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144710 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.144854 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144729 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-web-config\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.144854 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144756 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-config-volume\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.144854 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.144854 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.145129 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144897 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.145129 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sg2q\" (UniqueName: \"kubernetes.io/projected/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-kube-api-access-9sg2q\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.145129 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.145129 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.144988 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246048 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246016 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246048 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246052 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246285 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-web-config\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246285 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246230 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-config-volume\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246285 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246263 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246460 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246287 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246460 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246460 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246402 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sg2q\" (UniqueName: \"kubernetes.io/projected/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-kube-api-access-9sg2q\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246460 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246666 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246666 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246499 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-config-out\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246666 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246528 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246666 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.246922 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.246715 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.247692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.247663 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.248955 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.248932 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.249721 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.249360 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-web-config\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.249721 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.249582 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.249721 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.249594 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.249899 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.249878 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-config-volume\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.250196 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.250169 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.250325 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.250306 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.250694 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.250672 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.251035 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.251015 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.251107 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.251068 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-config-out\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.256862 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.256841 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sg2q\" (UniqueName: \"kubernetes.io/projected/7573c7b0-4f6b-44aa-8cba-a05da8df9fff-kube-api-access-9sg2q\") pod \"alertmanager-main-0\" (UID: \"7573c7b0-4f6b-44aa-8cba-a05da8df9fff\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.277799 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.277764 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7888c5fc-e91a-4621-8127-60b9447b59df" path="/var/lib/kubelet/pods/7888c5fc-e91a-4621-8127-60b9447b59df/volumes" Jul 10 14:47:48.355391 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.355310 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 14:47:48.502169 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.502135 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 14:47:48.505168 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:47:48.505140 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7573c7b0_4f6b_44aa_8cba_a05da8df9fff.slice/crio-ef5ed0cbf6d0412234e11aa78480ecb6c8dca535c027947bf2d56e755f5f208b WatchSource:0}: Error finding container ef5ed0cbf6d0412234e11aa78480ecb6c8dca535c027947bf2d56e755f5f208b: Status 404 returned error can't find the container with id ef5ed0cbf6d0412234e11aa78480ecb6c8dca535c027947bf2d56e755f5f208b Jul 10 14:47:48.860841 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.860807 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bb985b848-m4fb7"] Jul 10 14:47:48.862735 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.862715 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:48.877910 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.877884 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bb985b848-m4fb7"] Jul 10 14:47:48.951570 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.951536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-oauth-serving-cert\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:48.951570 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.951572 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-oauth-config\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:48.951784 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.951599 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f9g5\" (UniqueName: \"kubernetes.io/projected/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-kube-api-access-2f9g5\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:48.951784 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.951652 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-config\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:48.951784 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.951680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-serving-cert\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:48.951784 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.951693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-trusted-ca-bundle\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:48.951784 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.951712 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-service-ca\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:48.976130 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.976099 2562 generic.go:356] "Generic (PLEG): container finished" podID="7573c7b0-4f6b-44aa-8cba-a05da8df9fff" containerID="2be3d253dd9c2bc68a0af965381d8e2e8a049c17b4e42728b96c62f9dcd9899e" exitCode=0 Jul 10 14:47:48.976497 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.976190 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7573c7b0-4f6b-44aa-8cba-a05da8df9fff","Type":"ContainerDied","Data":"2be3d253dd9c2bc68a0af965381d8e2e8a049c17b4e42728b96c62f9dcd9899e"} Jul 10 14:47:48.976497 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:48.976261 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7573c7b0-4f6b-44aa-8cba-a05da8df9fff","Type":"ContainerStarted","Data":"ef5ed0cbf6d0412234e11aa78480ecb6c8dca535c027947bf2d56e755f5f208b"} Jul 10 14:47:49.052412 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.052294 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-config\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.052535 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.052410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-serving-cert\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.052535 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.052439 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-trusted-ca-bundle\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.052535 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.052468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-service-ca\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.052689 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.052661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-oauth-serving-cert\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.052740 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.052708 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-oauth-config\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.052792 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.052771 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f9g5\" (UniqueName: \"kubernetes.io/projected/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-kube-api-access-2f9g5\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.053411 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.052976 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-config\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.053725 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.053693 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-oauth-serving-cert\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.053815 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.053701 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-service-ca\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.053902 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.053881 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-trusted-ca-bundle\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.055466 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.055440 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-oauth-config\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.055806 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.055788 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-serving-cert\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.065449 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.065429 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f9g5\" (UniqueName: \"kubernetes.io/projected/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-kube-api-access-2f9g5\") pod \"console-7bb985b848-m4fb7\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.171573 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.171551 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:49.318050 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.318023 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bb985b848-m4fb7"] Jul 10 14:47:49.325477 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:47:49.325442 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cdd21dc_d580_4da1_b8bb_cf86dcc5080d.slice/crio-822423377d5e35050c04242341260f64b741fb22a15d7d9255ab28aba0e5b9e0 WatchSource:0}: Error finding container 822423377d5e35050c04242341260f64b741fb22a15d7d9255ab28aba0e5b9e0: Status 404 returned error can't find the container with id 822423377d5e35050c04242341260f64b741fb22a15d7d9255ab28aba0e5b9e0 Jul 10 14:47:49.981278 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.981241 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb985b848-m4fb7" event={"ID":"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d","Type":"ContainerStarted","Data":"7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441"} Jul 10 14:47:49.981820 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.981348 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb985b848-m4fb7" event={"ID":"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d","Type":"ContainerStarted","Data":"822423377d5e35050c04242341260f64b741fb22a15d7d9255ab28aba0e5b9e0"} Jul 10 14:47:49.985452 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.985424 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7573c7b0-4f6b-44aa-8cba-a05da8df9fff","Type":"ContainerStarted","Data":"df6064e99e0ca659713d66aa66b52f08ff2821ab05c46a5d5c41f59da7670c8c"} Jul 10 14:47:49.985564 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.985501 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7573c7b0-4f6b-44aa-8cba-a05da8df9fff","Type":"ContainerStarted","Data":"c054affeac3c8f80ecf7be404cb4c13f029548993412fc494f36746ed01649f5"} Jul 10 14:47:49.985564 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.985541 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7573c7b0-4f6b-44aa-8cba-a05da8df9fff","Type":"ContainerStarted","Data":"97a0961f7687ce525ccf9e9e7efdae04e1dd2bc731f37d7708c23b8cb9619cd8"} Jul 10 14:47:49.985633 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.985579 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7573c7b0-4f6b-44aa-8cba-a05da8df9fff","Type":"ContainerStarted","Data":"b8dcc3afe151ccc039ab96522998cc188fe95b985a8d499e71a8a267693d05c4"} Jul 10 14:47:49.985633 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.985622 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7573c7b0-4f6b-44aa-8cba-a05da8df9fff","Type":"ContainerStarted","Data":"5a5e6913328ca83b5195d30864284c4a0ec2370698c905e451f19fd443ae8f52"} Jul 10 14:47:49.985690 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:49.985650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7573c7b0-4f6b-44aa-8cba-a05da8df9fff","Type":"ContainerStarted","Data":"a99ae43895bc824ffff1896f4a0bf5833d453c6ff25cd7fb42d47605b58c653d"} Jul 10 14:47:50.022286 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:50.022234 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bb985b848-m4fb7" podStartSLOduration=2.022218037 podStartE2EDuration="2.022218037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:47:50.021348215 +0000 UTC m=+150.301862904" watchObservedRunningTime="2026-07-10 14:47:50.022218037 +0000 UTC m=+150.302732734" Jul 10 14:47:50.075091 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:50.075039 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.058616482 podStartE2EDuration="2.075024455s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.407973ms" isStatefulPod=false podCreationTimestamp="2026-07-10 14:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:47:50.074759469 +0000 UTC m=+150.355274173" watchObservedRunningTime="2026-07-10 14:47:50.075024455 +0000 UTC m=+150.355539153" Jul 10 14:47:59.171870 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:59.171830 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:59.172266 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:59.171913 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:47:59.176932 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:47:59.176910 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:48:00.029509 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:00.029478 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 14:48:00.114995 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:00.111315 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-578f6574db-fq7f8"] Jul 10 14:48:25.135076 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.135016 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-578f6574db-fq7f8" podUID="11660297-7d14-42a4-8829-28dfd5d8ecc8" containerName="console" containerID="cri-o://d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c" gracePeriod=15 Jul 10 14:48:25.381164 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.381135 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:48:25.464019 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.463931 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-serving-cert\") pod \"11660297-7d14-42a4-8829-28dfd5d8ecc8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " Jul 10 14:48:25.464019 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.463968 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-oauth-config\") pod \"11660297-7d14-42a4-8829-28dfd5d8ecc8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " Jul 10 14:48:25.464019 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.463993 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-config\") pod \"11660297-7d14-42a4-8829-28dfd5d8ecc8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " Jul 10 14:48:25.464019 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.464015 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5ltl\" (UniqueName: \"kubernetes.io/projected/11660297-7d14-42a4-8829-28dfd5d8ecc8-kube-api-access-l5ltl\") pod \"11660297-7d14-42a4-8829-28dfd5d8ecc8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " Jul 10 14:48:25.464368 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.464164 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-trusted-ca-bundle\") pod \"11660297-7d14-42a4-8829-28dfd5d8ecc8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " Jul 10 14:48:25.464368 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.464261 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-oauth-serving-cert\") pod \"11660297-7d14-42a4-8829-28dfd5d8ecc8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " Jul 10 14:48:25.464368 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.464292 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-service-ca\") pod \"11660297-7d14-42a4-8829-28dfd5d8ecc8\" (UID: \"11660297-7d14-42a4-8829-28dfd5d8ecc8\") " Jul 10 14:48:25.464583 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.464535 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-config" (OuterVolumeSpecName: "console-config") pod "11660297-7d14-42a4-8829-28dfd5d8ecc8" (UID: "11660297-7d14-42a4-8829-28dfd5d8ecc8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 14:48:25.464824 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.464646 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "11660297-7d14-42a4-8829-28dfd5d8ecc8" (UID: "11660297-7d14-42a4-8829-28dfd5d8ecc8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 14:48:25.464824 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.464691 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "11660297-7d14-42a4-8829-28dfd5d8ecc8" (UID: "11660297-7d14-42a4-8829-28dfd5d8ecc8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 14:48:25.465033 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.465008 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-service-ca" (OuterVolumeSpecName: "service-ca") pod "11660297-7d14-42a4-8829-28dfd5d8ecc8" (UID: "11660297-7d14-42a4-8829-28dfd5d8ecc8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 14:48:25.466392 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.466369 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "11660297-7d14-42a4-8829-28dfd5d8ecc8" (UID: "11660297-7d14-42a4-8829-28dfd5d8ecc8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:48:25.466392 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.466382 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "11660297-7d14-42a4-8829-28dfd5d8ecc8" (UID: "11660297-7d14-42a4-8829-28dfd5d8ecc8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:48:25.466506 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.466425 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11660297-7d14-42a4-8829-28dfd5d8ecc8-kube-api-access-l5ltl" (OuterVolumeSpecName: "kube-api-access-l5ltl") pod "11660297-7d14-42a4-8829-28dfd5d8ecc8" (UID: "11660297-7d14-42a4-8829-28dfd5d8ecc8"). InnerVolumeSpecName "kube-api-access-l5ltl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:48:25.564887 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.564857 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5ltl\" (UniqueName: \"kubernetes.io/projected/11660297-7d14-42a4-8829-28dfd5d8ecc8-kube-api-access-l5ltl\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:48:25.564887 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.564883 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-trusted-ca-bundle\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:48:25.564887 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.564892 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-oauth-serving-cert\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:48:25.565111 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.564903 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-service-ca\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:48:25.565111 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.564918 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-serving-cert\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:48:25.565111 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.564927 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-oauth-config\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:48:25.565111 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:25.564936 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11660297-7d14-42a4-8829-28dfd5d8ecc8-console-config\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:48:26.126041 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:26.126005 2562 generic.go:356] "Generic (PLEG): container finished" podID="11660297-7d14-42a4-8829-28dfd5d8ecc8" containerID="d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c" exitCode=0 Jul 10 14:48:26.126217 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:26.126055 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-578f6574db-fq7f8" Jul 10 14:48:26.126217 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:26.126068 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-578f6574db-fq7f8" event={"ID":"11660297-7d14-42a4-8829-28dfd5d8ecc8","Type":"ContainerDied","Data":"d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c"} Jul 10 14:48:26.126217 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:26.126120 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-578f6574db-fq7f8" event={"ID":"11660297-7d14-42a4-8829-28dfd5d8ecc8","Type":"ContainerDied","Data":"a65047be4cc89b553d451473c76ffb6cbd9c5b1d5fc6f54e22c1d83dbf67407b"} Jul 10 14:48:26.126217 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:26.126147 2562 scope.go:117] "RemoveContainer" containerID="d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c" Jul 10 14:48:26.134733 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:26.134710 2562 scope.go:117] "RemoveContainer" containerID="d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c" Jul 10 14:48:26.134986 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:48:26.134969 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c\": container with ID starting with d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c not found: ID does not exist" containerID="d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c" Jul 10 14:48:26.135036 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:26.134996 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c"} err="failed to get container status \"d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c\": rpc error: code = NotFound desc = could not find container \"d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c\": container with ID starting with d42dbb2176458aa364084195ba483a707d0236c95a3071653cc45c678b7b396c not found: ID does not exist" Jul 10 14:48:26.152526 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:26.152496 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-578f6574db-fq7f8"] Jul 10 14:48:26.160957 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:26.160935 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-578f6574db-fq7f8"] Jul 10 14:48:26.278117 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:26.278081 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11660297-7d14-42a4-8829-28dfd5d8ecc8" path="/var/lib/kubelet/pods/11660297-7d14-42a4-8829-28dfd5d8ecc8/volumes" Jul 10 14:48:53.356652 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.356615 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv"] Jul 10 14:48:53.357118 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.356900 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11660297-7d14-42a4-8829-28dfd5d8ecc8" containerName="console" Jul 10 14:48:53.357118 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.356910 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="11660297-7d14-42a4-8829-28dfd5d8ecc8" containerName="console" Jul 10 14:48:53.357118 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.356977 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="11660297-7d14-42a4-8829-28dfd5d8ecc8" containerName="console" Jul 10 14:48:53.358814 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.358798 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:48:53.361405 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.361385 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 14:48:53.361496 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.361440 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 14:48:53.362317 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.362300 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rmt7r\"" Jul 10 14:48:53.370778 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.370754 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv"] Jul 10 14:48:53.469422 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.469381 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:48:53.469599 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.469438 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2nmj\" (UniqueName: \"kubernetes.io/projected/1e231c86-a5f8-4e50-9588-5447d0e8762d-kube-api-access-c2nmj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:48:53.469599 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.469509 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:48:53.570775 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.570738 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:48:53.570878 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.570797 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2nmj\" (UniqueName: \"kubernetes.io/projected/1e231c86-a5f8-4e50-9588-5447d0e8762d-kube-api-access-c2nmj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:48:53.570878 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.570848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:48:53.571191 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.571175 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:48:53.571227 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.571198 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:48:53.580348 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.580311 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2nmj\" (UniqueName: \"kubernetes.io/projected/1e231c86-a5f8-4e50-9588-5447d0e8762d-kube-api-access-c2nmj\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:48:53.668149 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.668063 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:48:53.799739 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:53.799706 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv"] Jul 10 14:48:53.803007 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:48:53.802984 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e231c86_a5f8_4e50_9588_5447d0e8762d.slice/crio-aac2efc1edbf87171d1185b2ad53a641df1485ea0aa9ac0f7433692f3c414612 WatchSource:0}: Error finding container aac2efc1edbf87171d1185b2ad53a641df1485ea0aa9ac0f7433692f3c414612: Status 404 returned error can't find the container with id aac2efc1edbf87171d1185b2ad53a641df1485ea0aa9ac0f7433692f3c414612 Jul 10 14:48:54.234893 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:48:54.234857 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" event={"ID":"1e231c86-a5f8-4e50-9588-5447d0e8762d","Type":"ContainerStarted","Data":"aac2efc1edbf87171d1185b2ad53a641df1485ea0aa9ac0f7433692f3c414612"} Jul 10 14:49:02.269470 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:02.269425 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e231c86-a5f8-4e50-9588-5447d0e8762d" containerID="4a26518baac40b17cd6acef12646839b12e31a38dcab39a701d0ce29e95acaa2" exitCode=0 Jul 10 14:49:02.269848 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:02.269517 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" event={"ID":"1e231c86-a5f8-4e50-9588-5447d0e8762d","Type":"ContainerDied","Data":"4a26518baac40b17cd6acef12646839b12e31a38dcab39a701d0ce29e95acaa2"} Jul 10 14:49:05.282289 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:05.282258 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e231c86-a5f8-4e50-9588-5447d0e8762d" containerID="9bc69f3bbab8b7cf34d7f6d7ef87ba830ac971e4aa906df3d124112127bf61f9" exitCode=0 Jul 10 14:49:05.282799 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:05.282355 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" event={"ID":"1e231c86-a5f8-4e50-9588-5447d0e8762d","Type":"ContainerDied","Data":"9bc69f3bbab8b7cf34d7f6d7ef87ba830ac971e4aa906df3d124112127bf61f9"} Jul 10 14:49:14.319669 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:14.319633 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e231c86-a5f8-4e50-9588-5447d0e8762d" containerID="bb54436087668ed64577a0c16e90d206118322856ee28ce1af744cd5b5ce590d" exitCode=0 Jul 10 14:49:14.320109 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:14.319695 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" event={"ID":"1e231c86-a5f8-4e50-9588-5447d0e8762d","Type":"ContainerDied","Data":"bb54436087668ed64577a0c16e90d206118322856ee28ce1af744cd5b5ce590d"} Jul 10 14:49:15.493415 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:15.493379 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:49:15.663752 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:15.663672 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2nmj\" (UniqueName: \"kubernetes.io/projected/1e231c86-a5f8-4e50-9588-5447d0e8762d-kube-api-access-c2nmj\") pod \"1e231c86-a5f8-4e50-9588-5447d0e8762d\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " Jul 10 14:49:15.663904 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:15.663755 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-bundle\") pod \"1e231c86-a5f8-4e50-9588-5447d0e8762d\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " Jul 10 14:49:15.663904 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:15.663777 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-util\") pod \"1e231c86-a5f8-4e50-9588-5447d0e8762d\" (UID: \"1e231c86-a5f8-4e50-9588-5447d0e8762d\") " Jul 10 14:49:15.664465 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:15.664442 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-bundle" (OuterVolumeSpecName: "bundle") pod "1e231c86-a5f8-4e50-9588-5447d0e8762d" (UID: "1e231c86-a5f8-4e50-9588-5447d0e8762d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:49:15.666042 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:15.666016 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e231c86-a5f8-4e50-9588-5447d0e8762d-kube-api-access-c2nmj" (OuterVolumeSpecName: "kube-api-access-c2nmj") pod "1e231c86-a5f8-4e50-9588-5447d0e8762d" (UID: "1e231c86-a5f8-4e50-9588-5447d0e8762d"). InnerVolumeSpecName "kube-api-access-c2nmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:49:15.669419 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:15.669394 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-util" (OuterVolumeSpecName: "util") pod "1e231c86-a5f8-4e50-9588-5447d0e8762d" (UID: "1e231c86-a5f8-4e50-9588-5447d0e8762d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:49:15.764253 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:15.764222 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2nmj\" (UniqueName: \"kubernetes.io/projected/1e231c86-a5f8-4e50-9588-5447d0e8762d-kube-api-access-c2nmj\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:49:15.764253 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:15.764250 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-bundle\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:49:15.764253 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:15.764261 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e231c86-a5f8-4e50-9588-5447d0e8762d-util\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:49:16.329945 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:16.329915 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" Jul 10 14:49:16.329945 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:16.329936 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bbbzv" event={"ID":"1e231c86-a5f8-4e50-9588-5447d0e8762d","Type":"ContainerDied","Data":"aac2efc1edbf87171d1185b2ad53a641df1485ea0aa9ac0f7433692f3c414612"} Jul 10 14:49:16.330163 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:16.329985 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aac2efc1edbf87171d1185b2ad53a641df1485ea0aa9ac0f7433692f3c414612" Jul 10 14:49:21.418489 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.418447 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6"] Jul 10 14:49:21.419024 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.418825 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e231c86-a5f8-4e50-9588-5447d0e8762d" containerName="util" Jul 10 14:49:21.419024 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.418844 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e231c86-a5f8-4e50-9588-5447d0e8762d" containerName="util" Jul 10 14:49:21.419024 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.418853 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e231c86-a5f8-4e50-9588-5447d0e8762d" containerName="pull" Jul 10 14:49:21.419024 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.418861 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e231c86-a5f8-4e50-9588-5447d0e8762d" containerName="pull" Jul 10 14:49:21.419024 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.418910 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e231c86-a5f8-4e50-9588-5447d0e8762d" containerName="extract" Jul 10 14:49:21.419024 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.418920 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e231c86-a5f8-4e50-9588-5447d0e8762d" containerName="extract" Jul 10 14:49:21.419024 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.418990 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e231c86-a5f8-4e50-9588-5447d0e8762d" containerName="extract" Jul 10 14:49:21.421776 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.421755 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6" Jul 10 14:49:21.424311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.424286 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 10 14:49:21.424436 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.424321 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8wgh7\"" Jul 10 14:49:21.424436 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.424351 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 10 14:49:21.435541 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.435517 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6"] Jul 10 14:49:21.608745 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.608710 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-smtw6\" (UID: \"cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6" Jul 10 14:49:21.608745 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.608746 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gf2m\" (UniqueName: \"kubernetes.io/projected/cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4-kube-api-access-4gf2m\") pod \"cert-manager-operator-controller-manager-55c444bf86-smtw6\" (UID: \"cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6" Jul 10 14:49:21.709777 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.709698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-smtw6\" (UID: \"cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6" Jul 10 14:49:21.709777 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.709737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gf2m\" (UniqueName: \"kubernetes.io/projected/cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4-kube-api-access-4gf2m\") pod \"cert-manager-operator-controller-manager-55c444bf86-smtw6\" (UID: \"cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6" Jul 10 14:49:21.710069 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.710052 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-smtw6\" (UID: \"cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6" Jul 10 14:49:21.719374 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.719323 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gf2m\" (UniqueName: \"kubernetes.io/projected/cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4-kube-api-access-4gf2m\") pod \"cert-manager-operator-controller-manager-55c444bf86-smtw6\" (UID: \"cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6" Jul 10 14:49:21.731024 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.731004 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6" Jul 10 14:49:21.868228 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:49:21.868194 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9e8c34_4dd2_4f24_88f7_d2db4d3535d4.slice/crio-8a3300637e715faf213373886e0c45c22a275c250b2981d53033ba5fc95a15fa WatchSource:0}: Error finding container 8a3300637e715faf213373886e0c45c22a275c250b2981d53033ba5fc95a15fa: Status 404 returned error can't find the container with id 8a3300637e715faf213373886e0c45c22a275c250b2981d53033ba5fc95a15fa Jul 10 14:49:21.868398 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:21.868351 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6"] Jul 10 14:49:22.352212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:22.352177 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6" event={"ID":"cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4","Type":"ContainerStarted","Data":"8a3300637e715faf213373886e0c45c22a275c250b2981d53033ba5fc95a15fa"} Jul 10 14:49:24.365353 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:24.365293 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6" event={"ID":"cd9e8c34-4dd2-4f24-88f7-d2db4d3535d4","Type":"ContainerStarted","Data":"c97e02cd75c9aa16835f27dde73db68a81bdf81d241f2833b465093c46964dc3"} Jul 10 14:49:24.391747 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:24.391695 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-smtw6" podStartSLOduration=1.5719055480000002 podStartE2EDuration="3.391679169s" totalImagesPullingTime="1.819773621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:49:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:49:24.38994168 +0000 UTC m=+244.670456390" watchObservedRunningTime="2026-07-10 14:49:24.391679169 +0000 UTC m=+244.672193867" Jul 10 14:49:25.446185 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.446152 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph"] Jul 10 14:49:25.448387 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.448366 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:25.454595 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.454260 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 14:49:25.454595 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.454389 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rmt7r\"" Jul 10 14:49:25.454767 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.454748 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 14:49:25.471122 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.471093 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph"] Jul 10 14:49:25.639261 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.639207 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:25.639465 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.639277 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7phh\" (UniqueName: \"kubernetes.io/projected/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-kube-api-access-l7phh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:25.639465 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.639351 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:25.740014 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.739925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:25.740014 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.739975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7phh\" (UniqueName: \"kubernetes.io/projected/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-kube-api-access-l7phh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:25.740014 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.740006 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:25.740317 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.740301 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:25.740393 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.740374 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:25.744497 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.744474 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8l4dq"] Jul 10 14:49:25.746570 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.746555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" Jul 10 14:49:25.750959 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.750938 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-q69rr\"" Jul 10 14:49:25.751076 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.751056 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 14:49:25.751587 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.751572 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 14:49:25.752694 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.752673 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7phh\" (UniqueName: \"kubernetes.io/projected/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-kube-api-access-l7phh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:25.756986 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.756969 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:25.765384 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.765365 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8l4dq"] Jul 10 14:49:25.893736 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.893707 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph"] Jul 10 14:49:25.896950 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:49:25.896913 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe1dcf1a_cba6_49e2_ab40_a410ee58eced.slice/crio-f0c14e04d821863316a6e3845ec1f1b381faa3925a69ad7d811fafe454db0cc5 WatchSource:0}: Error finding container f0c14e04d821863316a6e3845ec1f1b381faa3925a69ad7d811fafe454db0cc5: Status 404 returned error can't find the container with id f0c14e04d821863316a6e3845ec1f1b381faa3925a69ad7d811fafe454db0cc5 Jul 10 14:49:25.941404 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.941380 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqs5t\" (UniqueName: \"kubernetes.io/projected/d67696d9-986f-47a4-ae9b-ab5dc3c62449-kube-api-access-wqs5t\") pod \"cert-manager-webhook-6486ffd997-8l4dq\" (UID: \"d67696d9-986f-47a4-ae9b-ab5dc3c62449\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" Jul 10 14:49:25.941540 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:25.941424 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d67696d9-986f-47a4-ae9b-ab5dc3c62449-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8l4dq\" (UID: \"d67696d9-986f-47a4-ae9b-ab5dc3c62449\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" Jul 10 14:49:26.042378 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:26.042319 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqs5t\" (UniqueName: \"kubernetes.io/projected/d67696d9-986f-47a4-ae9b-ab5dc3c62449-kube-api-access-wqs5t\") pod \"cert-manager-webhook-6486ffd997-8l4dq\" (UID: \"d67696d9-986f-47a4-ae9b-ab5dc3c62449\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" Jul 10 14:49:26.042510 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:26.042386 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d67696d9-986f-47a4-ae9b-ab5dc3c62449-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8l4dq\" (UID: \"d67696d9-986f-47a4-ae9b-ab5dc3c62449\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" Jul 10 14:49:26.051709 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:26.051686 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d67696d9-986f-47a4-ae9b-ab5dc3c62449-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8l4dq\" (UID: \"d67696d9-986f-47a4-ae9b-ab5dc3c62449\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" Jul 10 14:49:26.051847 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:26.051830 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqs5t\" (UniqueName: \"kubernetes.io/projected/d67696d9-986f-47a4-ae9b-ab5dc3c62449-kube-api-access-wqs5t\") pod \"cert-manager-webhook-6486ffd997-8l4dq\" (UID: \"d67696d9-986f-47a4-ae9b-ab5dc3c62449\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" Jul 10 14:49:26.071626 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:26.071608 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" Jul 10 14:49:26.196447 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:26.196420 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8l4dq"] Jul 10 14:49:26.199587 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:49:26.199561 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd67696d9_986f_47a4_ae9b_ab5dc3c62449.slice/crio-331fea9043b6c3d3ff671f850c53816a3da96d574e962719e8d11d98d3a3af13 WatchSource:0}: Error finding container 331fea9043b6c3d3ff671f850c53816a3da96d574e962719e8d11d98d3a3af13: Status 404 returned error can't find the container with id 331fea9043b6c3d3ff671f850c53816a3da96d574e962719e8d11d98d3a3af13 Jul 10 14:49:26.375271 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:26.375184 2562 generic.go:356] "Generic (PLEG): container finished" podID="fe1dcf1a-cba6-49e2-ab40-a410ee58eced" containerID="f5fd8cbaffbf2e2136a3b922a9faa805953d89dbcb244937753c261fad3f0c5b" exitCode=0 Jul 10 14:49:26.375431 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:26.375272 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" event={"ID":"fe1dcf1a-cba6-49e2-ab40-a410ee58eced","Type":"ContainerDied","Data":"f5fd8cbaffbf2e2136a3b922a9faa805953d89dbcb244937753c261fad3f0c5b"} Jul 10 14:49:26.375431 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:26.375345 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" event={"ID":"fe1dcf1a-cba6-49e2-ab40-a410ee58eced","Type":"ContainerStarted","Data":"f0c14e04d821863316a6e3845ec1f1b381faa3925a69ad7d811fafe454db0cc5"} Jul 10 14:49:26.376455 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:26.376430 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" event={"ID":"d67696d9-986f-47a4-ae9b-ab5dc3c62449","Type":"ContainerStarted","Data":"331fea9043b6c3d3ff671f850c53816a3da96d574e962719e8d11d98d3a3af13"} Jul 10 14:49:30.401636 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:30.401605 2562 generic.go:356] "Generic (PLEG): container finished" podID="fe1dcf1a-cba6-49e2-ab40-a410ee58eced" containerID="442193a44192dabba4e1d64ed2465a41f82101909c483361233360ee663ea908" exitCode=0 Jul 10 14:49:30.402089 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:30.401689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" event={"ID":"fe1dcf1a-cba6-49e2-ab40-a410ee58eced","Type":"ContainerDied","Data":"442193a44192dabba4e1d64ed2465a41f82101909c483361233360ee663ea908"} Jul 10 14:49:30.403242 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:30.403124 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" event={"ID":"d67696d9-986f-47a4-ae9b-ab5dc3c62449","Type":"ContainerStarted","Data":"bffde554e82e09325a9ccd7bb94b72b95e87d8fbdc7eedf65bffbb2903a5f4a0"} Jul 10 14:49:30.403303 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:30.403248 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" Jul 10 14:49:30.441221 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:30.441149 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" podStartSLOduration=2.183821681 podStartE2EDuration="5.441127764s" totalImagesPullingTime="3.257306083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:49:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:49:30.43894158 +0000 UTC m=+250.719456279" watchObservedRunningTime="2026-07-10 14:49:30.441127764 +0000 UTC m=+250.721642475" Jul 10 14:49:31.410152 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:31.410121 2562 generic.go:356] "Generic (PLEG): container finished" podID="fe1dcf1a-cba6-49e2-ab40-a410ee58eced" containerID="b41d0529c6bf39f1c5257957a5ceffc5b2406bd767f626db53da32682807ce3a" exitCode=0 Jul 10 14:49:31.410555 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:31.410198 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" event={"ID":"fe1dcf1a-cba6-49e2-ab40-a410ee58eced","Type":"ContainerDied","Data":"b41d0529c6bf39f1c5257957a5ceffc5b2406bd767f626db53da32682807ce3a"} Jul 10 14:49:32.539271 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.539246 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:32.599531 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.599502 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-util\") pod \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " Jul 10 14:49:32.599531 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.599537 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-bundle\") pod \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " Jul 10 14:49:32.599734 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.599595 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7phh\" (UniqueName: \"kubernetes.io/projected/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-kube-api-access-l7phh\") pod \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\" (UID: \"fe1dcf1a-cba6-49e2-ab40-a410ee58eced\") " Jul 10 14:49:32.599927 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.599904 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-bundle" (OuterVolumeSpecName: "bundle") pod "fe1dcf1a-cba6-49e2-ab40-a410ee58eced" (UID: "fe1dcf1a-cba6-49e2-ab40-a410ee58eced"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:49:32.601840 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.601817 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-kube-api-access-l7phh" (OuterVolumeSpecName: "kube-api-access-l7phh") pod "fe1dcf1a-cba6-49e2-ab40-a410ee58eced" (UID: "fe1dcf1a-cba6-49e2-ab40-a410ee58eced"). InnerVolumeSpecName "kube-api-access-l7phh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:49:32.604327 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.604295 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-util" (OuterVolumeSpecName: "util") pod "fe1dcf1a-cba6-49e2-ab40-a410ee58eced" (UID: "fe1dcf1a-cba6-49e2-ab40-a410ee58eced"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:49:32.700298 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.700225 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-bundle\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:49:32.700298 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.700250 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7phh\" (UniqueName: \"kubernetes.io/projected/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-kube-api-access-l7phh\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:49:32.700298 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.700262 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe1dcf1a-cba6-49e2-ab40-a410ee58eced-util\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:49:32.851923 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.851891 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-trt9b"] Jul 10 14:49:32.852182 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.852171 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe1dcf1a-cba6-49e2-ab40-a410ee58eced" containerName="util" Jul 10 14:49:32.852220 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.852183 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe1dcf1a-cba6-49e2-ab40-a410ee58eced" containerName="util" Jul 10 14:49:32.852220 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.852191 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe1dcf1a-cba6-49e2-ab40-a410ee58eced" containerName="pull" Jul 10 14:49:32.852220 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.852196 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe1dcf1a-cba6-49e2-ab40-a410ee58eced" containerName="pull" Jul 10 14:49:32.852220 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.852211 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe1dcf1a-cba6-49e2-ab40-a410ee58eced" containerName="extract" Jul 10 14:49:32.852220 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.852217 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe1dcf1a-cba6-49e2-ab40-a410ee58eced" containerName="extract" Jul 10 14:49:32.852390 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.852263 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe1dcf1a-cba6-49e2-ab40-a410ee58eced" containerName="extract" Jul 10 14:49:32.856112 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.856094 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-trt9b" Jul 10 14:49:32.858366 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.858321 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lx4zm\"" Jul 10 14:49:32.864708 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.864682 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-trt9b"] Jul 10 14:49:32.901354 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.901296 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eac9d9fa-25ed-482b-aa55-7274ce4da881-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-trt9b\" (UID: \"eac9d9fa-25ed-482b-aa55-7274ce4da881\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-trt9b" Jul 10 14:49:32.901522 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:32.901417 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp8fb\" (UniqueName: \"kubernetes.io/projected/eac9d9fa-25ed-482b-aa55-7274ce4da881-kube-api-access-xp8fb\") pod \"cert-manager-cainjector-559cdd8c87-trt9b\" (UID: \"eac9d9fa-25ed-482b-aa55-7274ce4da881\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-trt9b" Jul 10 14:49:33.002227 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.002189 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp8fb\" (UniqueName: \"kubernetes.io/projected/eac9d9fa-25ed-482b-aa55-7274ce4da881-kube-api-access-xp8fb\") pod \"cert-manager-cainjector-559cdd8c87-trt9b\" (UID: \"eac9d9fa-25ed-482b-aa55-7274ce4da881\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-trt9b" Jul 10 14:49:33.002420 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.002251 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eac9d9fa-25ed-482b-aa55-7274ce4da881-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-trt9b\" (UID: \"eac9d9fa-25ed-482b-aa55-7274ce4da881\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-trt9b" Jul 10 14:49:33.010742 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.010709 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp8fb\" (UniqueName: \"kubernetes.io/projected/eac9d9fa-25ed-482b-aa55-7274ce4da881-kube-api-access-xp8fb\") pod \"cert-manager-cainjector-559cdd8c87-trt9b\" (UID: \"eac9d9fa-25ed-482b-aa55-7274ce4da881\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-trt9b" Jul 10 14:49:33.011039 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.011017 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eac9d9fa-25ed-482b-aa55-7274ce4da881-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-trt9b\" (UID: \"eac9d9fa-25ed-482b-aa55-7274ce4da881\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-trt9b" Jul 10 14:49:33.165223 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.165193 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-trt9b" Jul 10 14:49:33.307879 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.307841 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-trt9b"] Jul 10 14:49:33.311916 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:49:33.311888 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeac9d9fa_25ed_482b_aa55_7274ce4da881.slice/crio-0e40bf296ecd49b877527e3bd17acd36824ca26440689abce5cc81fa92636a96 WatchSource:0}: Error finding container 0e40bf296ecd49b877527e3bd17acd36824ca26440689abce5cc81fa92636a96: Status 404 returned error can't find the container with id 0e40bf296ecd49b877527e3bd17acd36824ca26440689abce5cc81fa92636a96 Jul 10 14:49:33.422613 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.422586 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" Jul 10 14:49:33.422771 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.422587 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flsjph" event={"ID":"fe1dcf1a-cba6-49e2-ab40-a410ee58eced","Type":"ContainerDied","Data":"f0c14e04d821863316a6e3845ec1f1b381faa3925a69ad7d811fafe454db0cc5"} Jul 10 14:49:33.422771 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.422713 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0c14e04d821863316a6e3845ec1f1b381faa3925a69ad7d811fafe454db0cc5" Jul 10 14:49:33.424075 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.424053 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-trt9b" event={"ID":"eac9d9fa-25ed-482b-aa55-7274ce4da881","Type":"ContainerStarted","Data":"c33235143dcb3f73a222206d00218b469194b5bacc0b9c2f9bcf29cfb96f8738"} Jul 10 14:49:33.424075 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.424080 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-trt9b" event={"ID":"eac9d9fa-25ed-482b-aa55-7274ce4da881","Type":"ContainerStarted","Data":"0e40bf296ecd49b877527e3bd17acd36824ca26440689abce5cc81fa92636a96"} Jul 10 14:49:33.441286 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:33.441244 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-trt9b" podStartSLOduration=1.441229659 podStartE2EDuration="1.441229659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:49:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:49:33.439940963 +0000 UTC m=+253.720455662" watchObservedRunningTime="2026-07-10 14:49:33.441229659 +0000 UTC m=+253.721744357" Jul 10 14:49:36.412696 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:36.412671 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8l4dq" Jul 10 14:49:36.888706 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:36.888670 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-srsbc"] Jul 10 14:49:36.891961 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:36.891945 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-srsbc" Jul 10 14:49:36.894370 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:36.894325 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-v7pfk\"" Jul 10 14:49:36.899562 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:36.899543 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-srsbc"] Jul 10 14:49:36.933749 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:36.933725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcn9n\" (UniqueName: \"kubernetes.io/projected/4b5ebb58-eb1c-4e89-a249-69e945642ddf-kube-api-access-mcn9n\") pod \"cert-manager-55b4d954d5-srsbc\" (UID: \"4b5ebb58-eb1c-4e89-a249-69e945642ddf\") " pod="cert-manager/cert-manager-55b4d954d5-srsbc" Jul 10 14:49:36.933895 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:36.933754 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b5ebb58-eb1c-4e89-a249-69e945642ddf-bound-sa-token\") pod \"cert-manager-55b4d954d5-srsbc\" (UID: \"4b5ebb58-eb1c-4e89-a249-69e945642ddf\") " pod="cert-manager/cert-manager-55b4d954d5-srsbc" Jul 10 14:49:37.034756 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:37.034720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcn9n\" (UniqueName: \"kubernetes.io/projected/4b5ebb58-eb1c-4e89-a249-69e945642ddf-kube-api-access-mcn9n\") pod \"cert-manager-55b4d954d5-srsbc\" (UID: \"4b5ebb58-eb1c-4e89-a249-69e945642ddf\") " pod="cert-manager/cert-manager-55b4d954d5-srsbc" Jul 10 14:49:37.034756 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:37.034758 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b5ebb58-eb1c-4e89-a249-69e945642ddf-bound-sa-token\") pod \"cert-manager-55b4d954d5-srsbc\" (UID: \"4b5ebb58-eb1c-4e89-a249-69e945642ddf\") " pod="cert-manager/cert-manager-55b4d954d5-srsbc" Jul 10 14:49:37.045694 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:37.045667 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b5ebb58-eb1c-4e89-a249-69e945642ddf-bound-sa-token\") pod \"cert-manager-55b4d954d5-srsbc\" (UID: \"4b5ebb58-eb1c-4e89-a249-69e945642ddf\") " pod="cert-manager/cert-manager-55b4d954d5-srsbc" Jul 10 14:49:37.045879 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:37.045856 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcn9n\" (UniqueName: \"kubernetes.io/projected/4b5ebb58-eb1c-4e89-a249-69e945642ddf-kube-api-access-mcn9n\") pod \"cert-manager-55b4d954d5-srsbc\" (UID: \"4b5ebb58-eb1c-4e89-a249-69e945642ddf\") " pod="cert-manager/cert-manager-55b4d954d5-srsbc" Jul 10 14:49:37.202389 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:37.202271 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-srsbc" Jul 10 14:49:37.334614 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:37.334487 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-srsbc"] Jul 10 14:49:37.337186 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:49:37.337158 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b5ebb58_eb1c_4e89_a249_69e945642ddf.slice/crio-77989bffb2d1ff0cba0e4673d60664cb7b3da2ef1ffc4afd9cf2ed69895d9128 WatchSource:0}: Error finding container 77989bffb2d1ff0cba0e4673d60664cb7b3da2ef1ffc4afd9cf2ed69895d9128: Status 404 returned error can't find the container with id 77989bffb2d1ff0cba0e4673d60664cb7b3da2ef1ffc4afd9cf2ed69895d9128 Jul 10 14:49:37.442533 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:37.442496 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-srsbc" event={"ID":"4b5ebb58-eb1c-4e89-a249-69e945642ddf","Type":"ContainerStarted","Data":"1c3dd418835f191e0be1d47485caeaf96f82eab4070884c69f33543b33615758"} Jul 10 14:49:37.442877 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:37.442547 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-srsbc" event={"ID":"4b5ebb58-eb1c-4e89-a249-69e945642ddf","Type":"ContainerStarted","Data":"77989bffb2d1ff0cba0e4673d60664cb7b3da2ef1ffc4afd9cf2ed69895d9128"} Jul 10 14:49:37.460146 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:37.460032 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-srsbc" podStartSLOduration=1.460010284 podStartE2EDuration="1.460010284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:49:37.459683965 +0000 UTC m=+257.740198664" watchObservedRunningTime="2026-07-10 14:49:37.460010284 +0000 UTC m=+257.740524985" Jul 10 14:49:45.810428 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:45.810350 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l"] Jul 10 14:49:45.813224 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:45.813207 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:45.815972 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:45.815952 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rmt7r\"" Jul 10 14:49:45.816526 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:45.816511 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 14:49:45.816779 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:45.816768 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 14:49:45.827906 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:45.827881 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l"] Jul 10 14:49:45.907747 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:45.907718 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:45.907932 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:45.907773 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrcbt\" (UniqueName: \"kubernetes.io/projected/c0d022fd-184a-4491-a459-f3bb2ee40a5f-kube-api-access-jrcbt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:45.907932 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:45.907866 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:46.009069 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:46.009027 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:46.009274 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:46.009089 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrcbt\" (UniqueName: \"kubernetes.io/projected/c0d022fd-184a-4491-a459-f3bb2ee40a5f-kube-api-access-jrcbt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:46.009274 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:46.009118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:46.009561 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:46.009538 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:46.009561 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:46.009557 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:46.019002 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:46.018977 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrcbt\" (UniqueName: \"kubernetes.io/projected/c0d022fd-184a-4491-a459-f3bb2ee40a5f-kube-api-access-jrcbt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:46.122508 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:46.122420 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:46.250558 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:46.250522 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l"] Jul 10 14:49:46.252962 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:49:46.252935 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d022fd_184a_4491_a459_f3bb2ee40a5f.slice/crio-299324fe4ed9298f25f52fd595feceb01811920f50cd6aa2e0dd3084a0af2366 WatchSource:0}: Error finding container 299324fe4ed9298f25f52fd595feceb01811920f50cd6aa2e0dd3084a0af2366: Status 404 returned error can't find the container with id 299324fe4ed9298f25f52fd595feceb01811920f50cd6aa2e0dd3084a0af2366 Jul 10 14:49:46.484290 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:46.484197 2562 generic.go:356] "Generic (PLEG): container finished" podID="c0d022fd-184a-4491-a459-f3bb2ee40a5f" containerID="2617f88ce37366855c7fa2689e29612bf1aad8e1c36be34c7ea4d776250dc8c3" exitCode=0 Jul 10 14:49:46.484290 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:46.484276 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" event={"ID":"c0d022fd-184a-4491-a459-f3bb2ee40a5f","Type":"ContainerDied","Data":"2617f88ce37366855c7fa2689e29612bf1aad8e1c36be34c7ea4d776250dc8c3"} Jul 10 14:49:46.484494 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:46.484343 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" event={"ID":"c0d022fd-184a-4491-a459-f3bb2ee40a5f","Type":"ContainerStarted","Data":"299324fe4ed9298f25f52fd595feceb01811920f50cd6aa2e0dd3084a0af2366"} Jul 10 14:49:47.490562 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:47.490469 2562 generic.go:356] "Generic (PLEG): container finished" podID="c0d022fd-184a-4491-a459-f3bb2ee40a5f" containerID="292546f84a9b5b58e0a305cf42cc031d768e9e67a96ad72a73d59d76ed16f017" exitCode=0 Jul 10 14:49:47.490562 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:47.490551 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" event={"ID":"c0d022fd-184a-4491-a459-f3bb2ee40a5f","Type":"ContainerDied","Data":"292546f84a9b5b58e0a305cf42cc031d768e9e67a96ad72a73d59d76ed16f017"} Jul 10 14:49:48.496699 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:48.496665 2562 generic.go:356] "Generic (PLEG): container finished" podID="c0d022fd-184a-4491-a459-f3bb2ee40a5f" containerID="341aed9c611c63d95cf60feb183d73545a370eba7000dd77bfcfa35b51e72df9" exitCode=0 Jul 10 14:49:48.497085 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:48.496718 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" event={"ID":"c0d022fd-184a-4491-a459-f3bb2ee40a5f","Type":"ContainerDied","Data":"341aed9c611c63d95cf60feb183d73545a370eba7000dd77bfcfa35b51e72df9"} Jul 10 14:49:49.624007 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:49.623985 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:49.738089 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:49.738001 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrcbt\" (UniqueName: \"kubernetes.io/projected/c0d022fd-184a-4491-a459-f3bb2ee40a5f-kube-api-access-jrcbt\") pod \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " Jul 10 14:49:49.738253 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:49.738125 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-bundle\") pod \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " Jul 10 14:49:49.738253 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:49.738149 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-util\") pod \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\" (UID: \"c0d022fd-184a-4491-a459-f3bb2ee40a5f\") " Jul 10 14:49:49.738839 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:49.738808 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-bundle" (OuterVolumeSpecName: "bundle") pod "c0d022fd-184a-4491-a459-f3bb2ee40a5f" (UID: "c0d022fd-184a-4491-a459-f3bb2ee40a5f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:49:49.740124 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:49.740102 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0d022fd-184a-4491-a459-f3bb2ee40a5f-kube-api-access-jrcbt" (OuterVolumeSpecName: "kube-api-access-jrcbt") pod "c0d022fd-184a-4491-a459-f3bb2ee40a5f" (UID: "c0d022fd-184a-4491-a459-f3bb2ee40a5f"). InnerVolumeSpecName "kube-api-access-jrcbt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:49:49.743879 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:49.743845 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-util" (OuterVolumeSpecName: "util") pod "c0d022fd-184a-4491-a459-f3bb2ee40a5f" (UID: "c0d022fd-184a-4491-a459-f3bb2ee40a5f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:49:49.839699 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:49.839614 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrcbt\" (UniqueName: \"kubernetes.io/projected/c0d022fd-184a-4491-a459-f3bb2ee40a5f-kube-api-access-jrcbt\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:49:49.839699 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:49.839645 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-bundle\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:49:49.839699 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:49.839657 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0d022fd-184a-4491-a459-f3bb2ee40a5f-util\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:49:50.507203 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:50.507168 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" event={"ID":"c0d022fd-184a-4491-a459-f3bb2ee40a5f","Type":"ContainerDied","Data":"299324fe4ed9298f25f52fd595feceb01811920f50cd6aa2e0dd3084a0af2366"} Jul 10 14:49:50.507360 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:50.507219 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="299324fe4ed9298f25f52fd595feceb01811920f50cd6aa2e0dd3084a0af2366" Jul 10 14:49:50.507360 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:50.507179 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7nx5l" Jul 10 14:49:59.131820 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.131782 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd"] Jul 10 14:49:59.132282 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.132086 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d022fd-184a-4491-a459-f3bb2ee40a5f" containerName="extract" Jul 10 14:49:59.132282 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.132097 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d022fd-184a-4491-a459-f3bb2ee40a5f" containerName="extract" Jul 10 14:49:59.132282 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.132109 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d022fd-184a-4491-a459-f3bb2ee40a5f" containerName="util" Jul 10 14:49:59.132282 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.132115 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d022fd-184a-4491-a459-f3bb2ee40a5f" containerName="util" Jul 10 14:49:59.132282 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.132120 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d022fd-184a-4491-a459-f3bb2ee40a5f" containerName="pull" Jul 10 14:49:59.132282 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.132126 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d022fd-184a-4491-a459-f3bb2ee40a5f" containerName="pull" Jul 10 14:49:59.132282 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.132174 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d022fd-184a-4491-a459-f3bb2ee40a5f" containerName="extract" Jul 10 14:49:59.134565 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.134548 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.141016 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.140967 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 14:49:59.141385 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.141365 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vh6p9\"" Jul 10 14:49:59.142117 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.142103 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 14:49:59.142191 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.142137 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 14:49:59.143440 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.143422 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 14:49:59.156949 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.156923 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd"] Jul 10 14:49:59.221404 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.221368 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg9z6\" (UniqueName: \"kubernetes.io/projected/f9327d34-3718-4635-ae3f-8f7508fb0ee4-kube-api-access-zg9z6\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.221404 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.221408 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f9327d34-3718-4635-ae3f-8f7508fb0ee4-webhook-cert\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.221612 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.221433 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f9327d34-3718-4635-ae3f-8f7508fb0ee4-manifests\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.221612 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.221462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f9327d34-3718-4635-ae3f-8f7508fb0ee4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.221612 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.221508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f9327d34-3718-4635-ae3f-8f7508fb0ee4-tmp\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.322197 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.322154 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f9327d34-3718-4635-ae3f-8f7508fb0ee4-tmp\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.322419 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.322243 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg9z6\" (UniqueName: \"kubernetes.io/projected/f9327d34-3718-4635-ae3f-8f7508fb0ee4-kube-api-access-zg9z6\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.322419 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.322274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f9327d34-3718-4635-ae3f-8f7508fb0ee4-webhook-cert\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.322419 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.322315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f9327d34-3718-4635-ae3f-8f7508fb0ee4-manifests\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.322419 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.322366 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f9327d34-3718-4635-ae3f-8f7508fb0ee4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.322772 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.322741 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f9327d34-3718-4635-ae3f-8f7508fb0ee4-manifests\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.325145 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.325123 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f9327d34-3718-4635-ae3f-8f7508fb0ee4-tmp\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.325244 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.325203 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f9327d34-3718-4635-ae3f-8f7508fb0ee4-webhook-cert\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.325347 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.325314 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f9327d34-3718-4635-ae3f-8f7508fb0ee4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.330959 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.330936 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg9z6\" (UniqueName: \"kubernetes.io/projected/f9327d34-3718-4635-ae3f-8f7508fb0ee4-kube-api-access-zg9z6\") pod \"opendatahub-operator-controller-manager-666496fd47-hjhdd\" (UID: \"f9327d34-3718-4635-ae3f-8f7508fb0ee4\") " pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.445488 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.445406 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:49:59.586538 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:49:59.586414 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd"] Jul 10 14:50:00.555085 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:00.555004 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" event={"ID":"f9327d34-3718-4635-ae3f-8f7508fb0ee4","Type":"ContainerStarted","Data":"3ec0ce7aa45ddf1293025da2dd2c2caf1e177f437414909bc42b4bc338fb7036"} Jul 10 14:50:02.565636 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:02.565603 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" event={"ID":"f9327d34-3718-4635-ae3f-8f7508fb0ee4","Type":"ContainerStarted","Data":"9f2e0ca136fe091e8272e1eeb7d78987b993d73fbca2ca8fe2965f621bcff24e"} Jul 10 14:50:04.579031 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:04.578992 2562 generic.go:356] "Generic (PLEG): container finished" podID="f9327d34-3718-4635-ae3f-8f7508fb0ee4" containerID="9f2e0ca136fe091e8272e1eeb7d78987b993d73fbca2ca8fe2965f621bcff24e" exitCode=0 Jul 10 14:50:04.579657 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:04.579632 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" event={"ID":"f9327d34-3718-4635-ae3f-8f7508fb0ee4","Type":"ContainerDied","Data":"9f2e0ca136fe091e8272e1eeb7d78987b993d73fbca2ca8fe2965f621bcff24e"} Jul 10 14:50:05.586421 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:05.586382 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" event={"ID":"f9327d34-3718-4635-ae3f-8f7508fb0ee4","Type":"ContainerStarted","Data":"bcd3c081fd0d478933be9a2a07640b9cc0dac814877896f50a8cd8645cbf042e"} Jul 10 14:50:05.586833 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:05.586610 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:50:05.608961 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:05.608919 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" podStartSLOduration=2.323398555 podStartE2EDuration="6.608905053s" totalImagesPullingTime="3.275826318s" totalInitContainerRuntime="1.00968018s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:49:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:50:05.607997196 +0000 UTC m=+285.888511895" watchObservedRunningTime="2026-07-10 14:50:05.608905053 +0000 UTC m=+285.889419751" Jul 10 14:50:16.593144 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:16.593114 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-666496fd47-hjhdd" Jul 10 14:50:19.381838 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.381801 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p"] Jul 10 14:50:19.388104 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.388085 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" Jul 10 14:50:19.395572 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.395549 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 14:50:19.397049 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.397033 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 14:50:19.397107 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.397061 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 14:50:19.397225 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.397209 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 14:50:19.405186 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.405165 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14830b58-44e5-4e38-b2fa-2ef21983671b-tls-certs\") pod \"kube-auth-proxy-75598f7db4-fdh4p\" (UID: \"14830b58-44e5-4e38-b2fa-2ef21983671b\") " pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" Jul 10 14:50:19.405265 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.405190 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6psn6\" (UniqueName: \"kubernetes.io/projected/14830b58-44e5-4e38-b2fa-2ef21983671b-kube-api-access-6psn6\") pod \"kube-auth-proxy-75598f7db4-fdh4p\" (UID: \"14830b58-44e5-4e38-b2fa-2ef21983671b\") " pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" Jul 10 14:50:19.405304 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.405265 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14830b58-44e5-4e38-b2fa-2ef21983671b-tmp\") pod \"kube-auth-proxy-75598f7db4-fdh4p\" (UID: \"14830b58-44e5-4e38-b2fa-2ef21983671b\") " pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" Jul 10 14:50:19.453605 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.453572 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p"] Jul 10 14:50:19.505870 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.505837 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14830b58-44e5-4e38-b2fa-2ef21983671b-tmp\") pod \"kube-auth-proxy-75598f7db4-fdh4p\" (UID: \"14830b58-44e5-4e38-b2fa-2ef21983671b\") " pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" Jul 10 14:50:19.506034 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.505921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14830b58-44e5-4e38-b2fa-2ef21983671b-tls-certs\") pod \"kube-auth-proxy-75598f7db4-fdh4p\" (UID: \"14830b58-44e5-4e38-b2fa-2ef21983671b\") " pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" Jul 10 14:50:19.506034 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.505952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6psn6\" (UniqueName: \"kubernetes.io/projected/14830b58-44e5-4e38-b2fa-2ef21983671b-kube-api-access-6psn6\") pod \"kube-auth-proxy-75598f7db4-fdh4p\" (UID: \"14830b58-44e5-4e38-b2fa-2ef21983671b\") " pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" Jul 10 14:50:19.508187 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.508164 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14830b58-44e5-4e38-b2fa-2ef21983671b-tmp\") pod \"kube-auth-proxy-75598f7db4-fdh4p\" (UID: \"14830b58-44e5-4e38-b2fa-2ef21983671b\") " pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" Jul 10 14:50:19.508422 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.508404 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14830b58-44e5-4e38-b2fa-2ef21983671b-tls-certs\") pod \"kube-auth-proxy-75598f7db4-fdh4p\" (UID: \"14830b58-44e5-4e38-b2fa-2ef21983671b\") " pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" Jul 10 14:50:19.553764 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.553735 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6psn6\" (UniqueName: \"kubernetes.io/projected/14830b58-44e5-4e38-b2fa-2ef21983671b-kube-api-access-6psn6\") pod \"kube-auth-proxy-75598f7db4-fdh4p\" (UID: \"14830b58-44e5-4e38-b2fa-2ef21983671b\") " pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" Jul 10 14:50:19.697324 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.697242 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" Jul 10 14:50:19.850678 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.850644 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p"] Jul 10 14:50:19.853797 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:50:19.853765 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14830b58_44e5_4e38_b2fa_2ef21983671b.slice/crio-774b4edeb067f85940279a87c297f862cf6c463f18ba47c7725083c914f893a4 WatchSource:0}: Error finding container 774b4edeb067f85940279a87c297f862cf6c463f18ba47c7725083c914f893a4: Status 404 returned error can't find the container with id 774b4edeb067f85940279a87c297f862cf6c463f18ba47c7725083c914f893a4 Jul 10 14:50:19.931730 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.931697 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c8zx4"] Jul 10 14:50:19.934469 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.934454 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" Jul 10 14:50:19.941393 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.941370 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 10 14:50:19.941562 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.941376 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sb7t7\"" Jul 10 14:50:19.961269 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:19.961202 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c8zx4"] Jul 10 14:50:20.009590 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.009560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ea08f40-1256-453f-aff3-73749e4262e6-cert\") pod \"odh-model-controller-7998c67dc4-c8zx4\" (UID: \"8ea08f40-1256-453f-aff3-73749e4262e6\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" Jul 10 14:50:20.009751 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.009641 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7njjh\" (UniqueName: \"kubernetes.io/projected/8ea08f40-1256-453f-aff3-73749e4262e6-kube-api-access-7njjh\") pod \"odh-model-controller-7998c67dc4-c8zx4\" (UID: \"8ea08f40-1256-453f-aff3-73749e4262e6\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" Jul 10 14:50:20.110083 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.110049 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7njjh\" (UniqueName: \"kubernetes.io/projected/8ea08f40-1256-453f-aff3-73749e4262e6-kube-api-access-7njjh\") pod \"odh-model-controller-7998c67dc4-c8zx4\" (UID: \"8ea08f40-1256-453f-aff3-73749e4262e6\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" Jul 10 14:50:20.110270 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.110139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ea08f40-1256-453f-aff3-73749e4262e6-cert\") pod \"odh-model-controller-7998c67dc4-c8zx4\" (UID: \"8ea08f40-1256-453f-aff3-73749e4262e6\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" Jul 10 14:50:20.116929 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.116906 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 10 14:50:20.120388 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:50:20.120364 2562 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 10 14:50:20.120523 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:50:20.120442 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea08f40-1256-453f-aff3-73749e4262e6-cert podName:8ea08f40-1256-453f-aff3-73749e4262e6 nodeName:}" failed. No retries permitted until 2026-07-10 14:50:20.620420468 +0000 UTC m=+300.900935145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8ea08f40-1256-453f-aff3-73749e4262e6-cert") pod "odh-model-controller-7998c67dc4-c8zx4" (UID: "8ea08f40-1256-453f-aff3-73749e4262e6") : secret "odh-model-controller-webhook-cert" not found Jul 10 14:50:20.121478 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.121457 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 14:50:20.123622 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.123604 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 14:50:20.135435 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.135417 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 14:50:20.142691 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.142667 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7njjh\" (UniqueName: \"kubernetes.io/projected/8ea08f40-1256-453f-aff3-73749e4262e6-kube-api-access-7njjh\") pod \"odh-model-controller-7998c67dc4-c8zx4\" (UID: \"8ea08f40-1256-453f-aff3-73749e4262e6\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" Jul 10 14:50:20.656882 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.656833 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" event={"ID":"14830b58-44e5-4e38-b2fa-2ef21983671b","Type":"ContainerStarted","Data":"774b4edeb067f85940279a87c297f862cf6c463f18ba47c7725083c914f893a4"} Jul 10 14:50:20.714476 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.714436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ea08f40-1256-453f-aff3-73749e4262e6-cert\") pod \"odh-model-controller-7998c67dc4-c8zx4\" (UID: \"8ea08f40-1256-453f-aff3-73749e4262e6\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" Jul 10 14:50:20.717679 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.717652 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ea08f40-1256-453f-aff3-73749e4262e6-cert\") pod \"odh-model-controller-7998c67dc4-c8zx4\" (UID: \"8ea08f40-1256-453f-aff3-73749e4262e6\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" Jul 10 14:50:20.847250 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.847218 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sb7t7\"" Jul 10 14:50:20.855622 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.855592 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" Jul 10 14:50:20.997168 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:20.997143 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c8zx4"] Jul 10 14:50:20.999562 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:50:20.999535 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea08f40_1256_453f_aff3_73749e4262e6.slice/crio-96e60259527d55f1ea21d4510b48b81bbf0b4603494c178e1e4df2018bca1241 WatchSource:0}: Error finding container 96e60259527d55f1ea21d4510b48b81bbf0b4603494c178e1e4df2018bca1241: Status 404 returned error can't find the container with id 96e60259527d55f1ea21d4510b48b81bbf0b4603494c178e1e4df2018bca1241 Jul 10 14:50:21.152721 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.152679 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7zw2b"] Jul 10 14:50:21.157273 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.157220 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" Jul 10 14:50:21.160588 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.160501 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-6bllw\"" Jul 10 14:50:21.160588 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.160508 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 10 14:50:21.177838 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.177811 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7zw2b"] Jul 10 14:50:21.218702 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.218668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbl6d\" (UniqueName: \"kubernetes.io/projected/d0ca89ac-201e-4f30-9a1c-27e7dda107a2-kube-api-access-wbl6d\") pod \"model-serving-api-54d89bd5fc-7zw2b\" (UID: \"d0ca89ac-201e-4f30-9a1c-27e7dda107a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" Jul 10 14:50:21.218702 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.218705 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ca89ac-201e-4f30-9a1c-27e7dda107a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-7zw2b\" (UID: \"d0ca89ac-201e-4f30-9a1c-27e7dda107a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" Jul 10 14:50:21.319110 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.319071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbl6d\" (UniqueName: \"kubernetes.io/projected/d0ca89ac-201e-4f30-9a1c-27e7dda107a2-kube-api-access-wbl6d\") pod \"model-serving-api-54d89bd5fc-7zw2b\" (UID: \"d0ca89ac-201e-4f30-9a1c-27e7dda107a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" Jul 10 14:50:21.319273 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.319119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ca89ac-201e-4f30-9a1c-27e7dda107a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-7zw2b\" (UID: \"d0ca89ac-201e-4f30-9a1c-27e7dda107a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" Jul 10 14:50:21.319273 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:50:21.319245 2562 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 10 14:50:21.319440 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:50:21.319422 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0ca89ac-201e-4f30-9a1c-27e7dda107a2-tls-certs podName:d0ca89ac-201e-4f30-9a1c-27e7dda107a2 nodeName:}" failed. No retries permitted until 2026-07-10 14:50:21.819400646 +0000 UTC m=+302.099915328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d0ca89ac-201e-4f30-9a1c-27e7dda107a2-tls-certs") pod "model-serving-api-54d89bd5fc-7zw2b" (UID: "d0ca89ac-201e-4f30-9a1c-27e7dda107a2") : secret "model-serving-api-tls" not found Jul 10 14:50:21.348179 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.348141 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbl6d\" (UniqueName: \"kubernetes.io/projected/d0ca89ac-201e-4f30-9a1c-27e7dda107a2-kube-api-access-wbl6d\") pod \"model-serving-api-54d89bd5fc-7zw2b\" (UID: \"d0ca89ac-201e-4f30-9a1c-27e7dda107a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" Jul 10 14:50:21.572367 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.572319 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c24k4"] Jul 10 14:50:21.575656 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.575639 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.580860 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.580837 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 14:50:21.580964 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.580922 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zgscw\"" Jul 10 14:50:21.581193 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.581177 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 14:50:21.581319 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.581212 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 14:50:21.581512 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.581492 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 14:50:21.601163 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.601136 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c24k4"] Jul 10 14:50:21.621441 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.621415 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-token\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.621589 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.621444 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv6c2\" (UniqueName: \"kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-kube-api-access-kv6c2\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.621589 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.621493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2e0e6a2-7fb3-4bec-b52b-e48789609013-local-certs\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.621589 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.621548 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-cacerts\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.621589 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.621580 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.621751 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.621603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.621751 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.621647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.663022 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.662984 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" event={"ID":"8ea08f40-1256-453f-aff3-73749e4262e6","Type":"ContainerStarted","Data":"96e60259527d55f1ea21d4510b48b81bbf0b4603494c178e1e4df2018bca1241"} Jul 10 14:50:21.722564 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.722476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.722564 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.722557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-token\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.722794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.722581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv6c2\" (UniqueName: \"kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-kube-api-access-kv6c2\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.722794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.722645 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2e0e6a2-7fb3-4bec-b52b-e48789609013-local-certs\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.722794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.722680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-cacerts\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.722794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.722704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.722794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.722734 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.723929 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.723843 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.725903 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.725874 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.726171 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.726150 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2e0e6a2-7fb3-4bec-b52b-e48789609013-local-certs\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.726239 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.726194 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.726403 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.726381 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-cacerts\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.735533 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.735506 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv6c2\" (UniqueName: \"kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-kube-api-access-kv6c2\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.735668 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.735645 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-token\") pod \"istiod-openshift-gateway-7574459656-c24k4\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:21.823792 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.823664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ca89ac-201e-4f30-9a1c-27e7dda107a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-7zw2b\" (UID: \"d0ca89ac-201e-4f30-9a1c-27e7dda107a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" Jul 10 14:50:21.834321 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.834289 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ca89ac-201e-4f30-9a1c-27e7dda107a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-7zw2b\" (UID: \"d0ca89ac-201e-4f30-9a1c-27e7dda107a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" Jul 10 14:50:21.884931 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:21.884896 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:22.063015 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:22.062976 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c24k4"] Jul 10 14:50:22.067546 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:50:22.067515 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e0e6a2_7fb3_4bec_b52b_e48789609013.slice/crio-acc1f13539a5fe0a288c0eccac86be2ec9f8e2adc99113a7be36373d2a8b8822 WatchSource:0}: Error finding container acc1f13539a5fe0a288c0eccac86be2ec9f8e2adc99113a7be36373d2a8b8822: Status 404 returned error can't find the container with id acc1f13539a5fe0a288c0eccac86be2ec9f8e2adc99113a7be36373d2a8b8822 Jul 10 14:50:22.070514 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:22.070303 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" Jul 10 14:50:22.071010 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:22.070989 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 14:50:22.240387 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:22.236091 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7zw2b"] Jul 10 14:50:22.670848 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:22.670808 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" event={"ID":"e2e0e6a2-7fb3-4bec-b52b-e48789609013","Type":"ContainerStarted","Data":"acc1f13539a5fe0a288c0eccac86be2ec9f8e2adc99113a7be36373d2a8b8822"} Jul 10 14:50:23.056422 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:50:23.056382 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ca89ac_201e_4f30_9a1c_27e7dda107a2.slice/crio-f8e91e655031c7ef697528e88d05d5cb99a28dddd7e2801a472c34e6423cda10 WatchSource:0}: Error finding container f8e91e655031c7ef697528e88d05d5cb99a28dddd7e2801a472c34e6423cda10: Status 404 returned error can't find the container with id f8e91e655031c7ef697528e88d05d5cb99a28dddd7e2801a472c34e6423cda10 Jul 10 14:50:23.678822 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:23.678781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" event={"ID":"d0ca89ac-201e-4f30-9a1c-27e7dda107a2","Type":"ContainerStarted","Data":"f8e91e655031c7ef697528e88d05d5cb99a28dddd7e2801a472c34e6423cda10"} Jul 10 14:50:26.010172 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.010152 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 14:50:26.014092 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.014056 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 10 14:50:26.014181 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.014125 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 10 14:50:26.699088 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.699055 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" event={"ID":"8ea08f40-1256-453f-aff3-73749e4262e6","Type":"ContainerStarted","Data":"8676d2e92da68ecc1bfc146be124e46e953da5fd7f7fa7042267cfc8a2ec6f87"} Jul 10 14:50:26.699285 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.699149 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" Jul 10 14:50:26.701003 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.700978 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" event={"ID":"14830b58-44e5-4e38-b2fa-2ef21983671b","Type":"ContainerStarted","Data":"ffc0a7de3f91857714793dd1099449f3d7ac771a2c9a2e389a9e8f96ca7a2997"} Jul 10 14:50:26.702772 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.702748 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" event={"ID":"e2e0e6a2-7fb3-4bec-b52b-e48789609013","Type":"ContainerStarted","Data":"3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4"} Jul 10 14:50:26.702933 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.702913 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:26.704437 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.704412 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" event={"ID":"d0ca89ac-201e-4f30-9a1c-27e7dda107a2","Type":"ContainerStarted","Data":"b0b94cb8244721c3d6afcbdd120026adf08aa96f93fb498563b36da4f36bebaf"} Jul 10 14:50:26.704551 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.704535 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" Jul 10 14:50:26.761789 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.761723 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" podStartSLOduration=2.7554973240000002 podStartE2EDuration="7.761701339s" totalImagesPullingTime="5.006204015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:50:26.760850888 +0000 UTC m=+307.041365586" watchObservedRunningTime="2026-07-10 14:50:26.761701339 +0000 UTC m=+307.042216039" Jul 10 14:50:26.798126 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.798072 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" podStartSLOduration=2.840557799 podStartE2EDuration="5.798048405s" totalImagesPullingTime="2.957490606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:50:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:50:26.797366427 +0000 UTC m=+307.077881123" watchObservedRunningTime="2026-07-10 14:50:26.798048405 +0000 UTC m=+307.078563107" Jul 10 14:50:26.828021 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.827958 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-75598f7db4-fdh4p" podStartSLOduration=1.679609167 podStartE2EDuration="7.827937277s" totalImagesPullingTime="6.14832811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:50:26.825534032 +0000 UTC m=+307.106048732" watchObservedRunningTime="2026-07-10 14:50:26.827937277 +0000 UTC m=+307.108451978" Jul 10 14:50:26.864355 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:26.863558 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" podStartSLOduration=1.9208378320000001 podStartE2EDuration="5.863537174s" totalImagesPullingTime="3.942699342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:50:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:50:26.862438994 +0000 UTC m=+307.142953691" watchObservedRunningTime="2026-07-10 14:50:26.863537174 +0000 UTC m=+307.144051873" Jul 10 14:50:27.720250 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:27.720222 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:30.986956 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:30.986920 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7"] Jul 10 14:50:30.991594 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:30.991577 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" Jul 10 14:50:30.995062 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:30.995045 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 14:50:30.995160 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:30.995081 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bhtbd\"" Jul 10 14:50:31.007919 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.007890 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7"] Jul 10 14:50:31.023667 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.023637 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc"] Jul 10 14:50:31.027469 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.027446 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" Jul 10 14:50:31.029877 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.029852 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 10 14:50:31.030006 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.029945 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-m2bh9\"" Jul 10 14:50:31.045850 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.045828 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc"] Jul 10 14:50:31.109589 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.109551 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5q9m\" (UniqueName: \"kubernetes.io/projected/374aa998-1266-4f02-a4e2-578cebf7fc1d-kube-api-access-m5q9m\") pod \"llmisvc-controller-manager-777c6bd7d5-bctfc\" (UID: \"374aa998-1266-4f02-a4e2-578cebf7fc1d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" Jul 10 14:50:31.109767 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.109600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwm58\" (UniqueName: \"kubernetes.io/projected/86ba6845-0174-49b1-aed6-0d28e5820ee6-kube-api-access-bwm58\") pod \"kserve-controller-manager-d6f4b65b8-nk2g7\" (UID: \"86ba6845-0174-49b1-aed6-0d28e5820ee6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" Jul 10 14:50:31.109767 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.109627 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86ba6845-0174-49b1-aed6-0d28e5820ee6-cert\") pod \"kserve-controller-manager-d6f4b65b8-nk2g7\" (UID: \"86ba6845-0174-49b1-aed6-0d28e5820ee6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" Jul 10 14:50:31.109767 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.109648 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/374aa998-1266-4f02-a4e2-578cebf7fc1d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bctfc\" (UID: \"374aa998-1266-4f02-a4e2-578cebf7fc1d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" Jul 10 14:50:31.210825 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.210795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5q9m\" (UniqueName: \"kubernetes.io/projected/374aa998-1266-4f02-a4e2-578cebf7fc1d-kube-api-access-m5q9m\") pod \"llmisvc-controller-manager-777c6bd7d5-bctfc\" (UID: \"374aa998-1266-4f02-a4e2-578cebf7fc1d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" Jul 10 14:50:31.211030 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.210843 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwm58\" (UniqueName: \"kubernetes.io/projected/86ba6845-0174-49b1-aed6-0d28e5820ee6-kube-api-access-bwm58\") pod \"kserve-controller-manager-d6f4b65b8-nk2g7\" (UID: \"86ba6845-0174-49b1-aed6-0d28e5820ee6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" Jul 10 14:50:31.211030 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.210874 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86ba6845-0174-49b1-aed6-0d28e5820ee6-cert\") pod \"kserve-controller-manager-d6f4b65b8-nk2g7\" (UID: \"86ba6845-0174-49b1-aed6-0d28e5820ee6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" Jul 10 14:50:31.211030 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.210896 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/374aa998-1266-4f02-a4e2-578cebf7fc1d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bctfc\" (UID: \"374aa998-1266-4f02-a4e2-578cebf7fc1d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" Jul 10 14:50:31.211175 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:50:31.211051 2562 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 10 14:50:31.211175 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:50:31.211140 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/374aa998-1266-4f02-a4e2-578cebf7fc1d-cert podName:374aa998-1266-4f02-a4e2-578cebf7fc1d nodeName:}" failed. No retries permitted until 2026-07-10 14:50:31.711117258 +0000 UTC m=+311.991631936 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/374aa998-1266-4f02-a4e2-578cebf7fc1d-cert") pod "llmisvc-controller-manager-777c6bd7d5-bctfc" (UID: "374aa998-1266-4f02-a4e2-578cebf7fc1d") : secret "llmisvc-webhook-server-cert" not found Jul 10 14:50:31.213585 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.213562 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86ba6845-0174-49b1-aed6-0d28e5820ee6-cert\") pod \"kserve-controller-manager-d6f4b65b8-nk2g7\" (UID: \"86ba6845-0174-49b1-aed6-0d28e5820ee6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" Jul 10 14:50:31.225473 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.225449 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwm58\" (UniqueName: \"kubernetes.io/projected/86ba6845-0174-49b1-aed6-0d28e5820ee6-kube-api-access-bwm58\") pod \"kserve-controller-manager-d6f4b65b8-nk2g7\" (UID: \"86ba6845-0174-49b1-aed6-0d28e5820ee6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" Jul 10 14:50:31.228599 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.228577 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5q9m\" (UniqueName: \"kubernetes.io/projected/374aa998-1266-4f02-a4e2-578cebf7fc1d-kube-api-access-m5q9m\") pod \"llmisvc-controller-manager-777c6bd7d5-bctfc\" (UID: \"374aa998-1266-4f02-a4e2-578cebf7fc1d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" Jul 10 14:50:31.302855 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.302811 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" Jul 10 14:50:31.478749 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.478720 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7"] Jul 10 14:50:31.480537 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:50:31.480501 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ba6845_0174_49b1_aed6_0d28e5820ee6.slice/crio-dee107efbfa82405a4238126df322339693cf5d77789c059b4f40f762acb862e WatchSource:0}: Error finding container dee107efbfa82405a4238126df322339693cf5d77789c059b4f40f762acb862e: Status 404 returned error can't find the container with id dee107efbfa82405a4238126df322339693cf5d77789c059b4f40f762acb862e Jul 10 14:50:31.715606 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.715524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/374aa998-1266-4f02-a4e2-578cebf7fc1d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bctfc\" (UID: \"374aa998-1266-4f02-a4e2-578cebf7fc1d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" Jul 10 14:50:31.718015 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.717995 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/374aa998-1266-4f02-a4e2-578cebf7fc1d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bctfc\" (UID: \"374aa998-1266-4f02-a4e2-578cebf7fc1d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" Jul 10 14:50:31.739110 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.739083 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" event={"ID":"86ba6845-0174-49b1-aed6-0d28e5820ee6","Type":"ContainerStarted","Data":"dee107efbfa82405a4238126df322339693cf5d77789c059b4f40f762acb862e"} Jul 10 14:50:31.787894 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.787858 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c24k4"] Jul 10 14:50:31.788074 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.788054 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" podUID="e2e0e6a2-7fb3-4bec-b52b-e48789609013" containerName="discovery" containerID="cri-o://3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4" gracePeriod=30 Jul 10 14:50:31.937885 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:31.937841 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" Jul 10 14:50:32.035579 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.035553 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:32.090555 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.090526 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc"] Jul 10 14:50:32.093258 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:50:32.093235 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod374aa998_1266_4f02_a4e2_578cebf7fc1d.slice/crio-55d02a964e9ed53b3fdf1a4c39519f8fe5348baeca935dc9eef15787b39a2250 WatchSource:0}: Error finding container 55d02a964e9ed53b3fdf1a4c39519f8fe5348baeca935dc9eef15787b39a2250: Status 404 returned error can't find the container with id 55d02a964e9ed53b3fdf1a4c39519f8fe5348baeca935dc9eef15787b39a2250 Jul 10 14:50:32.118435 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.118406 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-cacerts\") pod \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " Jul 10 14:50:32.118544 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.118445 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-token\") pod \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " Jul 10 14:50:32.118544 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.118463 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-kubeconfig\") pod \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " Jul 10 14:50:32.118544 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.118507 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2e0e6a2-7fb3-4bec-b52b-e48789609013-local-certs\") pod \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " Jul 10 14:50:32.118704 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.118547 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv6c2\" (UniqueName: \"kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-kube-api-access-kv6c2\") pod \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " Jul 10 14:50:32.118704 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.118574 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-ca-configmap\") pod \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " Jul 10 14:50:32.118704 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.118600 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-dns-cert\") pod \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\" (UID: \"e2e0e6a2-7fb3-4bec-b52b-e48789609013\") " Jul 10 14:50:32.119442 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.119412 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e2e0e6a2-7fb3-4bec-b52b-e48789609013" (UID: "e2e0e6a2-7fb3-4bec-b52b-e48789609013"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 14:50:32.120870 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.120839 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e2e0e6a2-7fb3-4bec-b52b-e48789609013" (UID: "e2e0e6a2-7fb3-4bec-b52b-e48789609013"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:50:32.120974 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.120904 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2e0e6a2-7fb3-4bec-b52b-e48789609013" (UID: "e2e0e6a2-7fb3-4bec-b52b-e48789609013"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:50:32.121174 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.121159 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2e0e6a2-7fb3-4bec-b52b-e48789609013-local-certs" (OuterVolumeSpecName: "local-certs") pod "e2e0e6a2-7fb3-4bec-b52b-e48789609013" (UID: "e2e0e6a2-7fb3-4bec-b52b-e48789609013"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:50:32.121252 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.121237 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-kube-api-access-kv6c2" (OuterVolumeSpecName: "kube-api-access-kv6c2") pod "e2e0e6a2-7fb3-4bec-b52b-e48789609013" (UID: "e2e0e6a2-7fb3-4bec-b52b-e48789609013"). InnerVolumeSpecName "kube-api-access-kv6c2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:50:32.121292 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.121274 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-cacerts" (OuterVolumeSpecName: "cacerts") pod "e2e0e6a2-7fb3-4bec-b52b-e48789609013" (UID: "e2e0e6a2-7fb3-4bec-b52b-e48789609013"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:50:32.121504 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.121472 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e2e0e6a2-7fb3-4bec-b52b-e48789609013" (UID: "e2e0e6a2-7fb3-4bec-b52b-e48789609013"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:50:32.219665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.219633 2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-ca-configmap\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:50:32.219665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.219661 2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-csr-dns-cert\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:50:32.219665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.219671 2562 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-cacerts\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:50:32.219946 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.219681 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-token\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:50:32.219946 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.219691 2562 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2e0e6a2-7fb3-4bec-b52b-e48789609013-istio-kubeconfig\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:50:32.219946 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.219699 2562 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2e0e6a2-7fb3-4bec-b52b-e48789609013-local-certs\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:50:32.219946 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.219720 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv6c2\" (UniqueName: \"kubernetes.io/projected/e2e0e6a2-7fb3-4bec-b52b-e48789609013-kube-api-access-kv6c2\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:50:32.746893 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.746856 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" event={"ID":"374aa998-1266-4f02-a4e2-578cebf7fc1d","Type":"ContainerStarted","Data":"55d02a964e9ed53b3fdf1a4c39519f8fe5348baeca935dc9eef15787b39a2250"} Jul 10 14:50:32.748639 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.748596 2562 generic.go:356] "Generic (PLEG): container finished" podID="e2e0e6a2-7fb3-4bec-b52b-e48789609013" containerID="3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4" exitCode=0 Jul 10 14:50:32.748639 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.748634 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" event={"ID":"e2e0e6a2-7fb3-4bec-b52b-e48789609013","Type":"ContainerDied","Data":"3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4"} Jul 10 14:50:32.748846 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.748642 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" Jul 10 14:50:32.748846 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.748670 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c24k4" event={"ID":"e2e0e6a2-7fb3-4bec-b52b-e48789609013","Type":"ContainerDied","Data":"acc1f13539a5fe0a288c0eccac86be2ec9f8e2adc99113a7be36373d2a8b8822"} Jul 10 14:50:32.748846 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.748719 2562 scope.go:117] "RemoveContainer" containerID="3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4" Jul 10 14:50:32.771048 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.771027 2562 scope.go:117] "RemoveContainer" containerID="3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4" Jul 10 14:50:32.771397 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:50:32.771376 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4\": container with ID starting with 3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4 not found: ID does not exist" containerID="3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4" Jul 10 14:50:32.771481 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.771407 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4"} err="failed to get container status \"3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4\": rpc error: code = NotFound desc = could not find container \"3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4\": container with ID starting with 3274de60b32c338273a7558e4edf4134bebf3a18caa83b4d554bdf2b637509d4 not found: ID does not exist" Jul 10 14:50:32.781559 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.781367 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c24k4"] Jul 10 14:50:32.783853 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:32.783828 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c24k4"] Jul 10 14:50:34.280360 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:34.280317 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2e0e6a2-7fb3-4bec-b52b-e48789609013" path="/var/lib/kubelet/pods/e2e0e6a2-7fb3-4bec-b52b-e48789609013/volumes" Jul 10 14:50:34.763274 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:34.763237 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" event={"ID":"86ba6845-0174-49b1-aed6-0d28e5820ee6","Type":"ContainerStarted","Data":"9bdf879674605efbea90bf92e96df24b9a75ab49154811a3ba111539147d7fd6"} Jul 10 14:50:34.763477 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:34.763308 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" Jul 10 14:50:34.804685 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:34.804634 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" podStartSLOduration=1.884516209 podStartE2EDuration="4.804619705s" totalImagesPullingTime="2.920103496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:50:34.803586632 +0000 UTC m=+315.084101327" watchObservedRunningTime="2026-07-10 14:50:34.804619705 +0000 UTC m=+315.085134403" Jul 10 14:50:35.770291 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:35.770249 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" event={"ID":"374aa998-1266-4f02-a4e2-578cebf7fc1d","Type":"ContainerStarted","Data":"801e822fde0b7b7da7a4479dc801d4c47b56a9ad92ab5a9d9471ef04da0f27de"} Jul 10 14:50:35.770741 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:35.770390 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" Jul 10 14:50:35.803756 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:35.803705 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" podStartSLOduration=2.316676368 podStartE2EDuration="5.803690308s" totalImagesPullingTime="3.48701394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:50:35.799565405 +0000 UTC m=+316.080080103" watchObservedRunningTime="2026-07-10 14:50:35.803690308 +0000 UTC m=+316.084205006" Jul 10 14:50:37.720802 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:37.720775 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-c8zx4" Jul 10 14:50:37.722744 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:50:37.722727 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7zw2b" Jul 10 14:51:05.776134 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:05.776104 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nk2g7" Jul 10 14:51:06.777562 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:06.777530 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bctfc" Jul 10 14:51:20.481759 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.481680 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z8pc7"] Jul 10 14:51:20.482141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.482052 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2e0e6a2-7fb3-4bec-b52b-e48789609013" containerName="discovery" Jul 10 14:51:20.482141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.482064 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e0e6a2-7fb3-4bec-b52b-e48789609013" containerName="discovery" Jul 10 14:51:20.482141 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.482121 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2e0e6a2-7fb3-4bec-b52b-e48789609013" containerName="discovery" Jul 10 14:51:20.486280 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.486262 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" Jul 10 14:51:20.488733 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.488707 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 14:51:20.488897 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.488873 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 14:51:20.489819 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.489795 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-cz5zb\"" Jul 10 14:51:20.498914 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.498889 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z8pc7"] Jul 10 14:51:20.542249 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.542214 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24qtq\" (UniqueName: \"kubernetes.io/projected/35562d30-f425-4625-803c-81cd574a48d4-kube-api-access-24qtq\") pod \"kuadrant-operator-catalog-z8pc7\" (UID: \"35562d30-f425-4625-803c-81cd574a48d4\") " pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" Jul 10 14:51:20.643405 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.643356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24qtq\" (UniqueName: \"kubernetes.io/projected/35562d30-f425-4625-803c-81cd574a48d4-kube-api-access-24qtq\") pod \"kuadrant-operator-catalog-z8pc7\" (UID: \"35562d30-f425-4625-803c-81cd574a48d4\") " pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" Jul 10 14:51:20.651938 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.651900 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24qtq\" (UniqueName: \"kubernetes.io/projected/35562d30-f425-4625-803c-81cd574a48d4-kube-api-access-24qtq\") pod \"kuadrant-operator-catalog-z8pc7\" (UID: \"35562d30-f425-4625-803c-81cd574a48d4\") " pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" Jul 10 14:51:20.796444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.796409 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" Jul 10 14:51:20.836460 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.836431 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z8pc7"] Jul 10 14:51:20.927413 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.927389 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z8pc7"] Jul 10 14:51:20.929321 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:51:20.929291 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35562d30_f425_4625_803c_81cd574a48d4.slice/crio-fae6ca6724c5684e32cb32010079fba8f0b776e58b4d15280a7371617e67f95f WatchSource:0}: Error finding container fae6ca6724c5684e32cb32010079fba8f0b776e58b4d15280a7371617e67f95f: Status 404 returned error can't find the container with id fae6ca6724c5684e32cb32010079fba8f0b776e58b4d15280a7371617e67f95f Jul 10 14:51:20.988637 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:20.988601 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" event={"ID":"35562d30-f425-4625-803c-81cd574a48d4","Type":"ContainerStarted","Data":"fae6ca6724c5684e32cb32010079fba8f0b776e58b4d15280a7371617e67f95f"} Jul 10 14:51:21.044828 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:21.044787 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcszf"] Jul 10 14:51:21.048236 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:21.048178 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcszf" Jul 10 14:51:21.056549 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:21.056527 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcszf"] Jul 10 14:51:21.147542 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:21.147511 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bzhd\" (UniqueName: \"kubernetes.io/projected/53c37ada-cc91-4749-a306-229570685481-kube-api-access-4bzhd\") pod \"kuadrant-operator-catalog-pcszf\" (UID: \"53c37ada-cc91-4749-a306-229570685481\") " pod="kuadrant-system/kuadrant-operator-catalog-pcszf" Jul 10 14:51:21.248779 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:21.248728 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bzhd\" (UniqueName: \"kubernetes.io/projected/53c37ada-cc91-4749-a306-229570685481-kube-api-access-4bzhd\") pod \"kuadrant-operator-catalog-pcszf\" (UID: \"53c37ada-cc91-4749-a306-229570685481\") " pod="kuadrant-system/kuadrant-operator-catalog-pcszf" Jul 10 14:51:21.257840 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:21.257811 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bzhd\" (UniqueName: \"kubernetes.io/projected/53c37ada-cc91-4749-a306-229570685481-kube-api-access-4bzhd\") pod \"kuadrant-operator-catalog-pcszf\" (UID: \"53c37ada-cc91-4749-a306-229570685481\") " pod="kuadrant-system/kuadrant-operator-catalog-pcszf" Jul 10 14:51:21.358004 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:21.357913 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcszf" Jul 10 14:51:21.501501 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:21.501468 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcszf"] Jul 10 14:51:21.504645 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:51:21.504619 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c37ada_cc91_4749_a306_229570685481.slice/crio-238e2d3c012e00ce0e093c62f71780494073d13cf3171927873980accbdbae40 WatchSource:0}: Error finding container 238e2d3c012e00ce0e093c62f71780494073d13cf3171927873980accbdbae40: Status 404 returned error can't find the container with id 238e2d3c012e00ce0e093c62f71780494073d13cf3171927873980accbdbae40 Jul 10 14:51:21.996369 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:21.996315 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcszf" event={"ID":"53c37ada-cc91-4749-a306-229570685481","Type":"ContainerStarted","Data":"238e2d3c012e00ce0e093c62f71780494073d13cf3171927873980accbdbae40"} Jul 10 14:51:24.007869 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:24.007820 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcszf" event={"ID":"53c37ada-cc91-4749-a306-229570685481","Type":"ContainerStarted","Data":"d0d32d577cac5df92eb37383d2fdb76d5b2b9f80238908ead0545357241edc87"} Jul 10 14:51:24.009311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:24.009287 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" event={"ID":"35562d30-f425-4625-803c-81cd574a48d4","Type":"ContainerStarted","Data":"dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11"} Jul 10 14:51:24.009409 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:24.009357 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" podUID="35562d30-f425-4625-803c-81cd574a48d4" containerName="registry-server" containerID="cri-o://dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11" gracePeriod=2 Jul 10 14:51:24.026021 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:24.025978 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pcszf" podStartSLOduration=1.571721055 podStartE2EDuration="3.025962917s" totalImagesPullingTime="1.454241862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:51:24.023794545 +0000 UTC m=+364.304309244" watchObservedRunningTime="2026-07-10 14:51:24.025962917 +0000 UTC m=+364.306477615" Jul 10 14:51:24.042625 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:24.042578 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" podStartSLOduration=2.01597767 podStartE2EDuration="4.042563715s" totalImagesPullingTime="2.026586045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:51:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:51:24.041365714 +0000 UTC m=+364.321880411" watchObservedRunningTime="2026-07-10 14:51:24.042563715 +0000 UTC m=+364.323078413" Jul 10 14:51:24.255428 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:24.255405 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" Jul 10 14:51:24.375900 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:24.375810 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24qtq\" (UniqueName: \"kubernetes.io/projected/35562d30-f425-4625-803c-81cd574a48d4-kube-api-access-24qtq\") pod \"35562d30-f425-4625-803c-81cd574a48d4\" (UID: \"35562d30-f425-4625-803c-81cd574a48d4\") " Jul 10 14:51:24.378104 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:24.378068 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35562d30-f425-4625-803c-81cd574a48d4-kube-api-access-24qtq" (OuterVolumeSpecName: "kube-api-access-24qtq") pod "35562d30-f425-4625-803c-81cd574a48d4" (UID: "35562d30-f425-4625-803c-81cd574a48d4"). InnerVolumeSpecName "kube-api-access-24qtq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:51:24.477370 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:24.477312 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24qtq\" (UniqueName: \"kubernetes.io/projected/35562d30-f425-4625-803c-81cd574a48d4-kube-api-access-24qtq\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:25.015776 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:25.015743 2562 generic.go:356] "Generic (PLEG): container finished" podID="35562d30-f425-4625-803c-81cd574a48d4" containerID="dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11" exitCode=0 Jul 10 14:51:25.016295 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:25.015788 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" Jul 10 14:51:25.016295 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:25.015832 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" event={"ID":"35562d30-f425-4625-803c-81cd574a48d4","Type":"ContainerDied","Data":"dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11"} Jul 10 14:51:25.016295 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:25.015878 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z8pc7" event={"ID":"35562d30-f425-4625-803c-81cd574a48d4","Type":"ContainerDied","Data":"fae6ca6724c5684e32cb32010079fba8f0b776e58b4d15280a7371617e67f95f"} Jul 10 14:51:25.016295 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:25.015905 2562 scope.go:117] "RemoveContainer" containerID="dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11" Jul 10 14:51:25.025294 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:25.025276 2562 scope.go:117] "RemoveContainer" containerID="dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11" Jul 10 14:51:25.025595 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:51:25.025578 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11\": container with ID starting with dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11 not found: ID does not exist" containerID="dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11" Jul 10 14:51:25.025665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:25.025603 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11"} err="failed to get container status \"dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11\": rpc error: code = NotFound desc = could not find container \"dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11\": container with ID starting with dbd5022986cdf035d2396422c2f6aef9744065fe80b25d2e8a304156129dca11 not found: ID does not exist" Jul 10 14:51:25.042155 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:25.042130 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z8pc7"] Jul 10 14:51:25.047523 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:25.047504 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z8pc7"] Jul 10 14:51:26.279447 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:26.279409 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35562d30-f425-4625-803c-81cd574a48d4" path="/var/lib/kubelet/pods/35562d30-f425-4625-803c-81cd574a48d4/volumes" Jul 10 14:51:31.358359 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:31.358300 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-pcszf" Jul 10 14:51:31.358841 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:31.358384 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-pcszf" Jul 10 14:51:31.381291 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:31.381265 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-pcszf" Jul 10 14:51:32.073244 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:32.073214 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-pcszf" Jul 10 14:51:36.288643 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.288611 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d"] Jul 10 14:51:36.289096 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.288983 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35562d30-f425-4625-803c-81cd574a48d4" containerName="registry-server" Jul 10 14:51:36.289096 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.289001 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="35562d30-f425-4625-803c-81cd574a48d4" containerName="registry-server" Jul 10 14:51:36.289184 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.289099 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="35562d30-f425-4625-803c-81cd574a48d4" containerName="registry-server" Jul 10 14:51:36.292769 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.292753 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:36.295444 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.295423 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-77sqm\"" Jul 10 14:51:36.303561 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.303533 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d"] Jul 10 14:51:36.389417 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.389378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpwk2\" (UniqueName: \"kubernetes.io/projected/a6f7912d-a51c-4048-960c-5a72c447d47a-kube-api-access-wpwk2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:36.389710 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.389677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:36.389849 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.389760 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:36.490278 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.490240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:36.490278 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.490280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:36.490496 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.490323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpwk2\" (UniqueName: \"kubernetes.io/projected/a6f7912d-a51c-4048-960c-5a72c447d47a-kube-api-access-wpwk2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:36.490665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.490641 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:36.490701 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.490663 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:36.499400 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.499370 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpwk2\" (UniqueName: \"kubernetes.io/projected/a6f7912d-a51c-4048-960c-5a72c447d47a-kube-api-access-wpwk2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:36.601928 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.601851 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:36.737897 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:36.737873 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d"] Jul 10 14:51:36.742000 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:51:36.741961 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6f7912d_a51c_4048_960c_5a72c447d47a.slice/crio-bf0beedc5516652b5c05e443c950f30a75e725470e65ec8b15b7138bbb292014 WatchSource:0}: Error finding container bf0beedc5516652b5c05e443c950f30a75e725470e65ec8b15b7138bbb292014: Status 404 returned error can't find the container with id bf0beedc5516652b5c05e443c950f30a75e725470e65ec8b15b7138bbb292014 Jul 10 14:51:37.076083 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.076043 2562 generic.go:356] "Generic (PLEG): container finished" podID="a6f7912d-a51c-4048-960c-5a72c447d47a" containerID="eaa5b34e756c33a65849ee722055e46408c157da08f6712a2d5e024aa7691453" exitCode=0 Jul 10 14:51:37.076265 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.076105 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" event={"ID":"a6f7912d-a51c-4048-960c-5a72c447d47a","Type":"ContainerDied","Data":"eaa5b34e756c33a65849ee722055e46408c157da08f6712a2d5e024aa7691453"} Jul 10 14:51:37.076265 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.076148 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" event={"ID":"a6f7912d-a51c-4048-960c-5a72c447d47a","Type":"ContainerStarted","Data":"bf0beedc5516652b5c05e443c950f30a75e725470e65ec8b15b7138bbb292014"} Jul 10 14:51:37.086619 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.086544 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2"] Jul 10 14:51:37.089934 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.089916 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:37.099512 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.099483 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2"] Jul 10 14:51:37.195579 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.195526 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:37.195579 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.195586 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmkmg\" (UniqueName: \"kubernetes.io/projected/eb8996d9-2bd4-4342-8649-0a9161859e33-kube-api-access-nmkmg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:37.195806 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.195635 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:37.296171 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.296118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:37.296171 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.296175 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmkmg\" (UniqueName: \"kubernetes.io/projected/eb8996d9-2bd4-4342-8649-0a9161859e33-kube-api-access-nmkmg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:37.296619 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.296205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:37.296619 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.296592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:37.296619 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.296593 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:37.305096 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.305073 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmkmg\" (UniqueName: \"kubernetes.io/projected/eb8996d9-2bd4-4342-8649-0a9161859e33-kube-api-access-nmkmg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:37.400167 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.400088 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:37.493598 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.493564 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs"] Jul 10 14:51:37.499835 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.499811 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:37.507194 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.507090 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs"] Jul 10 14:51:37.549863 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.549836 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2"] Jul 10 14:51:37.552493 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:51:37.552462 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8996d9_2bd4_4342_8649_0a9161859e33.slice/crio-e3deeb08a5cd54a6ad8c0ed87cfecf907a44373dd567a567bffebf14758e1e47 WatchSource:0}: Error finding container e3deeb08a5cd54a6ad8c0ed87cfecf907a44373dd567a567bffebf14758e1e47: Status 404 returned error can't find the container with id e3deeb08a5cd54a6ad8c0ed87cfecf907a44373dd567a567bffebf14758e1e47 Jul 10 14:51:37.599919 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.599887 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:37.600080 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.599951 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5jhh\" (UniqueName: \"kubernetes.io/projected/77604509-3f54-4ab6-8c8f-7506b471507f-kube-api-access-r5jhh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:37.600080 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.600022 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:37.701417 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.701380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:37.701591 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.701498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5jhh\" (UniqueName: \"kubernetes.io/projected/77604509-3f54-4ab6-8c8f-7506b471507f-kube-api-access-r5jhh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:37.701591 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.701530 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:37.701830 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.701806 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:37.701903 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.701869 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:37.710533 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.710509 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5jhh\" (UniqueName: \"kubernetes.io/projected/77604509-3f54-4ab6-8c8f-7506b471507f-kube-api-access-r5jhh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:37.827657 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.827619 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:37.894431 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.894398 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn"] Jul 10 14:51:37.898877 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.898859 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:37.913258 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.913204 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn"] Jul 10 14:51:37.994222 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:37.994190 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs"] Jul 10 14:51:38.004095 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.004069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:38.004208 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.004123 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:38.004248 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.004208 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tjd4\" (UniqueName: \"kubernetes.io/projected/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-kube-api-access-6tjd4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:38.050520 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:51:38.050476 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77604509_3f54_4ab6_8c8f_7506b471507f.slice/crio-1b99fdbc64ac434ceabd8807fdec9d421130f10482592baf9b9a5a30c99aa32c WatchSource:0}: Error finding container 1b99fdbc64ac434ceabd8807fdec9d421130f10482592baf9b9a5a30c99aa32c: Status 404 returned error can't find the container with id 1b99fdbc64ac434ceabd8807fdec9d421130f10482592baf9b9a5a30c99aa32c Jul 10 14:51:38.082952 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.082924 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" event={"ID":"77604509-3f54-4ab6-8c8f-7506b471507f","Type":"ContainerStarted","Data":"1b99fdbc64ac434ceabd8807fdec9d421130f10482592baf9b9a5a30c99aa32c"} Jul 10 14:51:38.084771 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.084746 2562 generic.go:356] "Generic (PLEG): container finished" podID="a6f7912d-a51c-4048-960c-5a72c447d47a" containerID="24c3593ef324eaac4305a2c7971c530e10d36b14742fc6b0fcebf9eba6835dea" exitCode=0 Jul 10 14:51:38.084892 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.084805 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" event={"ID":"a6f7912d-a51c-4048-960c-5a72c447d47a","Type":"ContainerDied","Data":"24c3593ef324eaac4305a2c7971c530e10d36b14742fc6b0fcebf9eba6835dea"} Jul 10 14:51:38.086497 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.086473 2562 generic.go:356] "Generic (PLEG): container finished" podID="eb8996d9-2bd4-4342-8649-0a9161859e33" containerID="8e7fcdaf22e5c1c4b60fae0f23efba52f4f1cf1c19dece5a582ce99c1bd0b1a0" exitCode=0 Jul 10 14:51:38.086597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.086509 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" event={"ID":"eb8996d9-2bd4-4342-8649-0a9161859e33","Type":"ContainerDied","Data":"8e7fcdaf22e5c1c4b60fae0f23efba52f4f1cf1c19dece5a582ce99c1bd0b1a0"} Jul 10 14:51:38.086597 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.086544 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" event={"ID":"eb8996d9-2bd4-4342-8649-0a9161859e33","Type":"ContainerStarted","Data":"e3deeb08a5cd54a6ad8c0ed87cfecf907a44373dd567a567bffebf14758e1e47"} Jul 10 14:51:38.105164 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.105136 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:38.105276 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.105227 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tjd4\" (UniqueName: \"kubernetes.io/projected/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-kube-api-access-6tjd4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:38.105576 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.105279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:38.105635 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.105603 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:38.105635 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.105619 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:38.113363 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.113325 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tjd4\" (UniqueName: \"kubernetes.io/projected/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-kube-api-access-6tjd4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:38.214464 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.214438 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:38.356677 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:38.356644 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn"] Jul 10 14:51:38.359452 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:51:38.359429 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b8bd9ad_6a93_479d_8505_bf63a4598f3d.slice/crio-329fae82328ca31f6a150e415a00f75f8c272b319d807e5072229c639df1c934 WatchSource:0}: Error finding container 329fae82328ca31f6a150e415a00f75f8c272b319d807e5072229c639df1c934: Status 404 returned error can't find the container with id 329fae82328ca31f6a150e415a00f75f8c272b319d807e5072229c639df1c934 Jul 10 14:51:39.092516 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:39.092484 2562 generic.go:356] "Generic (PLEG): container finished" podID="77604509-3f54-4ab6-8c8f-7506b471507f" containerID="6e2dd27ed90e5cb30ff23e949358613054c6e277f26f70f5065633db52d880c5" exitCode=0 Jul 10 14:51:39.092702 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:39.092569 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" event={"ID":"77604509-3f54-4ab6-8c8f-7506b471507f","Type":"ContainerDied","Data":"6e2dd27ed90e5cb30ff23e949358613054c6e277f26f70f5065633db52d880c5"} Jul 10 14:51:39.094013 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:39.093990 2562 generic.go:356] "Generic (PLEG): container finished" podID="2b8bd9ad-6a93-479d-8505-bf63a4598f3d" containerID="2b2a97a3722534e2398a9c9d461b3396d0d540af71ff8ee9fc8676163e6574ad" exitCode=0 Jul 10 14:51:39.094101 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:39.094065 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" event={"ID":"2b8bd9ad-6a93-479d-8505-bf63a4598f3d","Type":"ContainerDied","Data":"2b2a97a3722534e2398a9c9d461b3396d0d540af71ff8ee9fc8676163e6574ad"} Jul 10 14:51:39.094158 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:39.094114 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" event={"ID":"2b8bd9ad-6a93-479d-8505-bf63a4598f3d","Type":"ContainerStarted","Data":"329fae82328ca31f6a150e415a00f75f8c272b319d807e5072229c639df1c934"} Jul 10 14:51:39.096377 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:39.096357 2562 generic.go:356] "Generic (PLEG): container finished" podID="a6f7912d-a51c-4048-960c-5a72c447d47a" containerID="d76dfeb620877ed3cb3c46e5841e49589b56c906690eec32e9d4b54637037fc9" exitCode=0 Jul 10 14:51:39.096464 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:39.096422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" event={"ID":"a6f7912d-a51c-4048-960c-5a72c447d47a","Type":"ContainerDied","Data":"d76dfeb620877ed3cb3c46e5841e49589b56c906690eec32e9d4b54637037fc9"} Jul 10 14:51:39.098197 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:39.098171 2562 generic.go:356] "Generic (PLEG): container finished" podID="eb8996d9-2bd4-4342-8649-0a9161859e33" containerID="33b0590a77f7ffe88f4a3a741db1b10756f489237dc83107d7545cf9956f5bb5" exitCode=0 Jul 10 14:51:39.098269 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:39.098210 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" event={"ID":"eb8996d9-2bd4-4342-8649-0a9161859e33","Type":"ContainerDied","Data":"33b0590a77f7ffe88f4a3a741db1b10756f489237dc83107d7545cf9956f5bb5"} Jul 10 14:51:40.105312 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.105278 2562 generic.go:356] "Generic (PLEG): container finished" podID="eb8996d9-2bd4-4342-8649-0a9161859e33" containerID="64e021bafd71bcba46588a9c990e26c0cc32b3ea306ca4cfaf88c81016a77561" exitCode=0 Jul 10 14:51:40.105716 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.105363 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" event={"ID":"eb8996d9-2bd4-4342-8649-0a9161859e33","Type":"ContainerDied","Data":"64e021bafd71bcba46588a9c990e26c0cc32b3ea306ca4cfaf88c81016a77561"} Jul 10 14:51:40.246823 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.246790 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:40.425846 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.425738 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpwk2\" (UniqueName: \"kubernetes.io/projected/a6f7912d-a51c-4048-960c-5a72c447d47a-kube-api-access-wpwk2\") pod \"a6f7912d-a51c-4048-960c-5a72c447d47a\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " Jul 10 14:51:40.425846 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.425808 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-util\") pod \"a6f7912d-a51c-4048-960c-5a72c447d47a\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " Jul 10 14:51:40.425846 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.425829 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-bundle\") pod \"a6f7912d-a51c-4048-960c-5a72c447d47a\" (UID: \"a6f7912d-a51c-4048-960c-5a72c447d47a\") " Jul 10 14:51:40.426326 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.426302 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-bundle" (OuterVolumeSpecName: "bundle") pod "a6f7912d-a51c-4048-960c-5a72c447d47a" (UID: "a6f7912d-a51c-4048-960c-5a72c447d47a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:51:40.427990 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.427967 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6f7912d-a51c-4048-960c-5a72c447d47a-kube-api-access-wpwk2" (OuterVolumeSpecName: "kube-api-access-wpwk2") pod "a6f7912d-a51c-4048-960c-5a72c447d47a" (UID: "a6f7912d-a51c-4048-960c-5a72c447d47a"). InnerVolumeSpecName "kube-api-access-wpwk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:51:40.430947 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.430924 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-util" (OuterVolumeSpecName: "util") pod "a6f7912d-a51c-4048-960c-5a72c447d47a" (UID: "a6f7912d-a51c-4048-960c-5a72c447d47a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:51:40.527176 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.527129 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpwk2\" (UniqueName: \"kubernetes.io/projected/a6f7912d-a51c-4048-960c-5a72c447d47a-kube-api-access-wpwk2\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:40.527176 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.527170 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-util\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:40.527176 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:40.527180 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6f7912d-a51c-4048-960c-5a72c447d47a-bundle\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:41.112130 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.112098 2562 generic.go:356] "Generic (PLEG): container finished" podID="77604509-3f54-4ab6-8c8f-7506b471507f" containerID="7d39ca494bd1bb89478301c88cb2b0388c3f34225acb4ab4905cab47feade64b" exitCode=0 Jul 10 14:51:41.112557 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.112191 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" event={"ID":"77604509-3f54-4ab6-8c8f-7506b471507f","Type":"ContainerDied","Data":"7d39ca494bd1bb89478301c88cb2b0388c3f34225acb4ab4905cab47feade64b"} Jul 10 14:51:41.114045 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.114021 2562 generic.go:356] "Generic (PLEG): container finished" podID="2b8bd9ad-6a93-479d-8505-bf63a4598f3d" containerID="27cb3e88d05d4be9a71e704b3cb9b8e5362d8351f18994836d9c49fb3b50842c" exitCode=0 Jul 10 14:51:41.114125 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.114102 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" event={"ID":"2b8bd9ad-6a93-479d-8505-bf63a4598f3d","Type":"ContainerDied","Data":"27cb3e88d05d4be9a71e704b3cb9b8e5362d8351f18994836d9c49fb3b50842c"} Jul 10 14:51:41.116121 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.116105 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" Jul 10 14:51:41.116199 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.116132 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d" event={"ID":"a6f7912d-a51c-4048-960c-5a72c447d47a","Type":"ContainerDied","Data":"bf0beedc5516652b5c05e443c950f30a75e725470e65ec8b15b7138bbb292014"} Jul 10 14:51:41.116199 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.116161 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf0beedc5516652b5c05e443c950f30a75e725470e65ec8b15b7138bbb292014" Jul 10 14:51:41.271797 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.271757 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:41.435154 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.435126 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-util\") pod \"eb8996d9-2bd4-4342-8649-0a9161859e33\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " Jul 10 14:51:41.435318 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.435207 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmkmg\" (UniqueName: \"kubernetes.io/projected/eb8996d9-2bd4-4342-8649-0a9161859e33-kube-api-access-nmkmg\") pod \"eb8996d9-2bd4-4342-8649-0a9161859e33\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " Jul 10 14:51:41.435318 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.435275 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-bundle\") pod \"eb8996d9-2bd4-4342-8649-0a9161859e33\" (UID: \"eb8996d9-2bd4-4342-8649-0a9161859e33\") " Jul 10 14:51:41.435753 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.435723 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-bundle" (OuterVolumeSpecName: "bundle") pod "eb8996d9-2bd4-4342-8649-0a9161859e33" (UID: "eb8996d9-2bd4-4342-8649-0a9161859e33"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:51:41.437378 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.437353 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8996d9-2bd4-4342-8649-0a9161859e33-kube-api-access-nmkmg" (OuterVolumeSpecName: "kube-api-access-nmkmg") pod "eb8996d9-2bd4-4342-8649-0a9161859e33" (UID: "eb8996d9-2bd4-4342-8649-0a9161859e33"). InnerVolumeSpecName "kube-api-access-nmkmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:51:41.440748 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.440723 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-util" (OuterVolumeSpecName: "util") pod "eb8996d9-2bd4-4342-8649-0a9161859e33" (UID: "eb8996d9-2bd4-4342-8649-0a9161859e33"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:51:41.535997 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.535905 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmkmg\" (UniqueName: \"kubernetes.io/projected/eb8996d9-2bd4-4342-8649-0a9161859e33-kube-api-access-nmkmg\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:41.535997 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.535940 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-bundle\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:41.535997 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:41.535950 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb8996d9-2bd4-4342-8649-0a9161859e33-util\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:42.123504 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:42.123468 2562 generic.go:356] "Generic (PLEG): container finished" podID="77604509-3f54-4ab6-8c8f-7506b471507f" containerID="2fe15fa7bab466122666527650fa027ea319f63377dc8eb2876a4f4b8fd792a2" exitCode=0 Jul 10 14:51:42.123898 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:42.123552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" event={"ID":"77604509-3f54-4ab6-8c8f-7506b471507f","Type":"ContainerDied","Data":"2fe15fa7bab466122666527650fa027ea319f63377dc8eb2876a4f4b8fd792a2"} Jul 10 14:51:42.125532 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:42.125510 2562 generic.go:356] "Generic (PLEG): container finished" podID="2b8bd9ad-6a93-479d-8505-bf63a4598f3d" containerID="f3b9e861c5eeccc3c78f7f6f6c66d07ed224a931bf6b2e926edbd1d108a646a9" exitCode=0 Jul 10 14:51:42.125649 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:42.125554 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" event={"ID":"2b8bd9ad-6a93-479d-8505-bf63a4598f3d","Type":"ContainerDied","Data":"f3b9e861c5eeccc3c78f7f6f6c66d07ed224a931bf6b2e926edbd1d108a646a9"} Jul 10 14:51:42.127367 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:42.127351 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" Jul 10 14:51:42.127435 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:42.127380 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2" event={"ID":"eb8996d9-2bd4-4342-8649-0a9161859e33","Type":"ContainerDied","Data":"e3deeb08a5cd54a6ad8c0ed87cfecf907a44373dd567a567bffebf14758e1e47"} Jul 10 14:51:42.127435 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:42.127414 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3deeb08a5cd54a6ad8c0ed87cfecf907a44373dd567a567bffebf14758e1e47" Jul 10 14:51:43.289750 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.289727 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:43.291638 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.291616 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:43.451649 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.451565 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tjd4\" (UniqueName: \"kubernetes.io/projected/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-kube-api-access-6tjd4\") pod \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " Jul 10 14:51:43.451649 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.451613 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5jhh\" (UniqueName: \"kubernetes.io/projected/77604509-3f54-4ab6-8c8f-7506b471507f-kube-api-access-r5jhh\") pod \"77604509-3f54-4ab6-8c8f-7506b471507f\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " Jul 10 14:51:43.451649 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.451644 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-bundle\") pod \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " Jul 10 14:51:43.451881 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.451707 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-util\") pod \"77604509-3f54-4ab6-8c8f-7506b471507f\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " Jul 10 14:51:43.451881 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.451754 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-bundle\") pod \"77604509-3f54-4ab6-8c8f-7506b471507f\" (UID: \"77604509-3f54-4ab6-8c8f-7506b471507f\") " Jul 10 14:51:43.451881 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.451825 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-util\") pod \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\" (UID: \"2b8bd9ad-6a93-479d-8505-bf63a4598f3d\") " Jul 10 14:51:43.452399 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.452370 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-bundle" (OuterVolumeSpecName: "bundle") pod "2b8bd9ad-6a93-479d-8505-bf63a4598f3d" (UID: "2b8bd9ad-6a93-479d-8505-bf63a4598f3d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:51:43.452550 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.452521 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-bundle" (OuterVolumeSpecName: "bundle") pod "77604509-3f54-4ab6-8c8f-7506b471507f" (UID: "77604509-3f54-4ab6-8c8f-7506b471507f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:51:43.453923 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.453904 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77604509-3f54-4ab6-8c8f-7506b471507f-kube-api-access-r5jhh" (OuterVolumeSpecName: "kube-api-access-r5jhh") pod "77604509-3f54-4ab6-8c8f-7506b471507f" (UID: "77604509-3f54-4ab6-8c8f-7506b471507f"). InnerVolumeSpecName "kube-api-access-r5jhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:51:43.454206 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.454188 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-kube-api-access-6tjd4" (OuterVolumeSpecName: "kube-api-access-6tjd4") pod "2b8bd9ad-6a93-479d-8505-bf63a4598f3d" (UID: "2b8bd9ad-6a93-479d-8505-bf63a4598f3d"). InnerVolumeSpecName "kube-api-access-6tjd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:51:43.457462 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.457442 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-util" (OuterVolumeSpecName: "util") pod "77604509-3f54-4ab6-8c8f-7506b471507f" (UID: "77604509-3f54-4ab6-8c8f-7506b471507f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:51:43.460592 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.460564 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-util" (OuterVolumeSpecName: "util") pod "2b8bd9ad-6a93-479d-8505-bf63a4598f3d" (UID: "2b8bd9ad-6a93-479d-8505-bf63a4598f3d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:51:43.553386 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.553359 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tjd4\" (UniqueName: \"kubernetes.io/projected/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-kube-api-access-6tjd4\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:43.553386 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.553381 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5jhh\" (UniqueName: \"kubernetes.io/projected/77604509-3f54-4ab6-8c8f-7506b471507f-kube-api-access-r5jhh\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:43.553386 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.553392 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-bundle\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:43.553608 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.553400 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-util\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:43.553608 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.553410 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77604509-3f54-4ab6-8c8f-7506b471507f-bundle\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:43.553608 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:43.553417 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b8bd9ad-6a93-479d-8505-bf63a4598f3d-util\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:51:44.139815 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:44.139785 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" Jul 10 14:51:44.139815 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:44.139790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs" event={"ID":"77604509-3f54-4ab6-8c8f-7506b471507f","Type":"ContainerDied","Data":"1b99fdbc64ac434ceabd8807fdec9d421130f10482592baf9b9a5a30c99aa32c"} Jul 10 14:51:44.140039 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:44.139840 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b99fdbc64ac434ceabd8807fdec9d421130f10482592baf9b9a5a30c99aa32c" Jul 10 14:51:44.141669 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:44.141643 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" Jul 10 14:51:44.141808 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:44.141648 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn" event={"ID":"2b8bd9ad-6a93-479d-8505-bf63a4598f3d","Type":"ContainerDied","Data":"329fae82328ca31f6a150e415a00f75f8c272b319d807e5072229c639df1c934"} Jul 10 14:51:44.141808 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:44.141758 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="329fae82328ca31f6a150e415a00f75f8c272b319d807e5072229c639df1c934" Jul 10 14:51:54.596196 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596161 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk"] Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596515 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6f7912d-a51c-4048-960c-5a72c447d47a" containerName="extract" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596528 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6f7912d-a51c-4048-960c-5a72c447d47a" containerName="extract" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596535 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77604509-3f54-4ab6-8c8f-7506b471507f" containerName="extract" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596541 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77604509-3f54-4ab6-8c8f-7506b471507f" containerName="extract" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596547 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b8bd9ad-6a93-479d-8505-bf63a4598f3d" containerName="extract" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596552 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b8bd9ad-6a93-479d-8505-bf63a4598f3d" containerName="extract" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596560 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b8bd9ad-6a93-479d-8505-bf63a4598f3d" containerName="util" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596565 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b8bd9ad-6a93-479d-8505-bf63a4598f3d" containerName="util" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596573 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb8996d9-2bd4-4342-8649-0a9161859e33" containerName="extract" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596578 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb8996d9-2bd4-4342-8649-0a9161859e33" containerName="extract" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596585 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77604509-3f54-4ab6-8c8f-7506b471507f" containerName="pull" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596591 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77604509-3f54-4ab6-8c8f-7506b471507f" containerName="pull" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596602 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb8996d9-2bd4-4342-8649-0a9161859e33" containerName="pull" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596609 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb8996d9-2bd4-4342-8649-0a9161859e33" containerName="pull" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596626 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b8bd9ad-6a93-479d-8505-bf63a4598f3d" containerName="pull" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596633 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b8bd9ad-6a93-479d-8505-bf63a4598f3d" containerName="pull" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596656 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb8996d9-2bd4-4342-8649-0a9161859e33" containerName="util" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596663 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb8996d9-2bd4-4342-8649-0a9161859e33" containerName="util" Jul 10 14:51:54.596665 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596671 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77604509-3f54-4ab6-8c8f-7506b471507f" containerName="util" Jul 10 14:51:54.597212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596679 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77604509-3f54-4ab6-8c8f-7506b471507f" containerName="util" Jul 10 14:51:54.597212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596692 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6f7912d-a51c-4048-960c-5a72c447d47a" containerName="util" Jul 10 14:51:54.597212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596700 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6f7912d-a51c-4048-960c-5a72c447d47a" containerName="util" Jul 10 14:51:54.597212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596710 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6f7912d-a51c-4048-960c-5a72c447d47a" containerName="pull" Jul 10 14:51:54.597212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596718 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6f7912d-a51c-4048-960c-5a72c447d47a" containerName="pull" Jul 10 14:51:54.597212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596803 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="77604509-3f54-4ab6-8c8f-7506b471507f" containerName="extract" Jul 10 14:51:54.597212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596815 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b8bd9ad-6a93-479d-8505-bf63a4598f3d" containerName="extract" Jul 10 14:51:54.597212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596826 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6f7912d-a51c-4048-960c-5a72c447d47a" containerName="extract" Jul 10 14:51:54.597212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.596834 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb8996d9-2bd4-4342-8649-0a9161859e33" containerName="extract" Jul 10 14:51:54.601044 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.601027 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:51:54.603517 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.603481 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fffgr\"" Jul 10 14:51:54.613847 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.613823 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk"] Jul 10 14:51:54.641828 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.641800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6227887-18dd-4108-8605-1d3888080933-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" (UID: \"f6227887-18dd-4108-8605-1d3888080933\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:51:54.641949 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.641857 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7qpd\" (UniqueName: \"kubernetes.io/projected/f6227887-18dd-4108-8605-1d3888080933-kube-api-access-n7qpd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" (UID: \"f6227887-18dd-4108-8605-1d3888080933\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:51:54.742877 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.742843 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7qpd\" (UniqueName: \"kubernetes.io/projected/f6227887-18dd-4108-8605-1d3888080933-kube-api-access-n7qpd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" (UID: \"f6227887-18dd-4108-8605-1d3888080933\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:51:54.743095 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.742918 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6227887-18dd-4108-8605-1d3888080933-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" (UID: \"f6227887-18dd-4108-8605-1d3888080933\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:51:54.743260 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.743243 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6227887-18dd-4108-8605-1d3888080933-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" (UID: \"f6227887-18dd-4108-8605-1d3888080933\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:51:54.757326 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.757303 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7qpd\" (UniqueName: \"kubernetes.io/projected/f6227887-18dd-4108-8605-1d3888080933-kube-api-access-n7qpd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" (UID: \"f6227887-18dd-4108-8605-1d3888080933\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:51:54.911163 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:54.911058 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:51:55.052913 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:55.052888 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk"] Jul 10 14:51:55.055268 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:51:55.055239 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6227887_18dd_4108_8605_1d3888080933.slice/crio-862413ead6a3ac246605938c74665769ad49f1708edbae87642619ca8daf78d8 WatchSource:0}: Error finding container 862413ead6a3ac246605938c74665769ad49f1708edbae87642619ca8daf78d8: Status 404 returned error can't find the container with id 862413ead6a3ac246605938c74665769ad49f1708edbae87642619ca8daf78d8 Jul 10 14:51:55.199051 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:55.198969 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" event={"ID":"f6227887-18dd-4108-8605-1d3888080933","Type":"ContainerStarted","Data":"862413ead6a3ac246605938c74665769ad49f1708edbae87642619ca8daf78d8"} Jul 10 14:51:57.202958 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:57.202914 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-82csm"] Jul 10 14:51:57.209301 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:57.209281 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-82csm" Jul 10 14:51:57.218714 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:57.218693 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-4gv4l\"" Jul 10 14:51:57.228969 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:57.228944 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-82csm"] Jul 10 14:51:57.265097 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:57.265059 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6xxd\" (UniqueName: \"kubernetes.io/projected/7806869e-eb08-4dd1-ad59-ff3cbb288ddb-kube-api-access-s6xxd\") pod \"authorino-operator-75cc8d5cb7-82csm\" (UID: \"7806869e-eb08-4dd1-ad59-ff3cbb288ddb\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-82csm" Jul 10 14:51:57.365865 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:57.365818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6xxd\" (UniqueName: \"kubernetes.io/projected/7806869e-eb08-4dd1-ad59-ff3cbb288ddb-kube-api-access-s6xxd\") pod \"authorino-operator-75cc8d5cb7-82csm\" (UID: \"7806869e-eb08-4dd1-ad59-ff3cbb288ddb\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-82csm" Jul 10 14:51:57.385428 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:57.385402 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6xxd\" (UniqueName: \"kubernetes.io/projected/7806869e-eb08-4dd1-ad59-ff3cbb288ddb-kube-api-access-s6xxd\") pod \"authorino-operator-75cc8d5cb7-82csm\" (UID: \"7806869e-eb08-4dd1-ad59-ff3cbb288ddb\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-82csm" Jul 10 14:51:57.519686 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:57.519650 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-82csm" Jul 10 14:51:57.687381 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:57.687358 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-82csm"] Jul 10 14:51:57.688783 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:51:57.688750 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7806869e_eb08_4dd1_ad59_ff3cbb288ddb.slice/crio-1d81aa6174270b9f40bff3343a62525b200fed34bafae72d35c16db7b3862879 WatchSource:0}: Error finding container 1d81aa6174270b9f40bff3343a62525b200fed34bafae72d35c16db7b3862879: Status 404 returned error can't find the container with id 1d81aa6174270b9f40bff3343a62525b200fed34bafae72d35c16db7b3862879 Jul 10 14:51:58.220992 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:58.220951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-82csm" event={"ID":"7806869e-eb08-4dd1-ad59-ff3cbb288ddb","Type":"ContainerStarted","Data":"1d81aa6174270b9f40bff3343a62525b200fed34bafae72d35c16db7b3862879"} Jul 10 14:51:59.501402 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:59.501363 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn"] Jul 10 14:51:59.509621 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:59.509594 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn" Jul 10 14:51:59.512499 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:59.512478 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-mtswr\"" Jul 10 14:51:59.512632 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:59.512506 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 10 14:51:59.519676 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:59.519648 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn"] Jul 10 14:51:59.587078 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:59.587036 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5rmd\" (UniqueName: \"kubernetes.io/projected/dd2650cd-cd7f-44d5-89c2-93c06eb657cb-kube-api-access-n5rmd\") pod \"dns-operator-controller-manager-864dd78c4c-49tqn\" (UID: \"dd2650cd-cd7f-44d5-89c2-93c06eb657cb\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn" Jul 10 14:51:59.688198 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:59.688164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5rmd\" (UniqueName: \"kubernetes.io/projected/dd2650cd-cd7f-44d5-89c2-93c06eb657cb-kube-api-access-n5rmd\") pod \"dns-operator-controller-manager-864dd78c4c-49tqn\" (UID: \"dd2650cd-cd7f-44d5-89c2-93c06eb657cb\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn" Jul 10 14:51:59.700058 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:59.699995 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5rmd\" (UniqueName: \"kubernetes.io/projected/dd2650cd-cd7f-44d5-89c2-93c06eb657cb-kube-api-access-n5rmd\") pod \"dns-operator-controller-manager-864dd78c4c-49tqn\" (UID: \"dd2650cd-cd7f-44d5-89c2-93c06eb657cb\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn" Jul 10 14:51:59.823684 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:51:59.823610 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn" Jul 10 14:52:00.311286 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:00.311260 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn"] Jul 10 14:52:00.313372 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:52:00.313324 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd2650cd_cd7f_44d5_89c2_93c06eb657cb.slice/crio-d17e3de44007afc6c1410ab47e6eaef58c5a34f5e1a6f240b60d60697375c733 WatchSource:0}: Error finding container d17e3de44007afc6c1410ab47e6eaef58c5a34f5e1a6f240b60d60697375c733: Status 404 returned error can't find the container with id d17e3de44007afc6c1410ab47e6eaef58c5a34f5e1a6f240b60d60697375c733 Jul 10 14:52:01.240804 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.240758 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn" event={"ID":"dd2650cd-cd7f-44d5-89c2-93c06eb657cb","Type":"ContainerStarted","Data":"d17e3de44007afc6c1410ab47e6eaef58c5a34f5e1a6f240b60d60697375c733"} Jul 10 14:52:01.242609 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.242580 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-82csm" event={"ID":"7806869e-eb08-4dd1-ad59-ff3cbb288ddb","Type":"ContainerStarted","Data":"475dda1c36c14d4bfa8e8871cbdc82e24fb9da4e53424a5864a2435fd1f66738"} Jul 10 14:52:01.242768 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.242745 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-82csm" Jul 10 14:52:01.267085 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.267016 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-82csm" podStartSLOduration=1.739838472 podStartE2EDuration="4.26699328s" totalImagesPullingTime="2.527154808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:51:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:52:01.263607166 +0000 UTC m=+401.544121861" watchObservedRunningTime="2026-07-10 14:52:01.26699328 +0000 UTC m=+401.547507979" Jul 10 14:52:01.538590 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.538424 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk"] Jul 10 14:52:01.546877 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.546851 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:01.550269 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.550245 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-trfwb\"" Jul 10 14:52:01.559283 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.559247 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk"] Jul 10 14:52:01.609496 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.609456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqjft\" (UniqueName: \"kubernetes.io/projected/8c580aab-8c72-4662-91bf-6accbe55177d-kube-api-access-nqjft\") pod \"limitador-operator-controller-manager-859d7b78bd-mkmrk\" (UID: \"8c580aab-8c72-4662-91bf-6accbe55177d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:01.710495 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.710450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqjft\" (UniqueName: \"kubernetes.io/projected/8c580aab-8c72-4662-91bf-6accbe55177d-kube-api-access-nqjft\") pod \"limitador-operator-controller-manager-859d7b78bd-mkmrk\" (UID: \"8c580aab-8c72-4662-91bf-6accbe55177d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:01.720069 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.720038 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqjft\" (UniqueName: \"kubernetes.io/projected/8c580aab-8c72-4662-91bf-6accbe55177d-kube-api-access-nqjft\") pod \"limitador-operator-controller-manager-859d7b78bd-mkmrk\" (UID: \"8c580aab-8c72-4662-91bf-6accbe55177d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:01.860112 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:01.860022 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:02.023984 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:02.023960 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk"] Jul 10 14:52:02.025928 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:52:02.025906 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c580aab_8c72_4662_91bf_6accbe55177d.slice/crio-61252de54fa7c483ed3adb324dfe49f73ab1fe5550b9fe5a17983a83ad47197a WatchSource:0}: Error finding container 61252de54fa7c483ed3adb324dfe49f73ab1fe5550b9fe5a17983a83ad47197a: Status 404 returned error can't find the container with id 61252de54fa7c483ed3adb324dfe49f73ab1fe5550b9fe5a17983a83ad47197a Jul 10 14:52:02.250250 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:02.250216 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" event={"ID":"f6227887-18dd-4108-8605-1d3888080933","Type":"ContainerStarted","Data":"8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8"} Jul 10 14:52:02.250786 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:02.250378 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:02.251655 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:02.251635 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" event={"ID":"8c580aab-8c72-4662-91bf-6accbe55177d","Type":"ContainerStarted","Data":"61252de54fa7c483ed3adb324dfe49f73ab1fe5550b9fe5a17983a83ad47197a"} Jul 10 14:52:02.275375 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:02.275298 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" podStartSLOduration=1.385584581 podStartE2EDuration="8.275275237s" totalImagesPullingTime="6.889690656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:52:02.27236704 +0000 UTC m=+402.552881735" watchObservedRunningTime="2026-07-10 14:52:02.275275237 +0000 UTC m=+402.555789938" Jul 10 14:52:04.266746 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:04.266629 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" event={"ID":"8c580aab-8c72-4662-91bf-6accbe55177d","Type":"ContainerStarted","Data":"390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79"} Jul 10 14:52:04.266746 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:04.266720 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:04.268584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:04.268554 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn" event={"ID":"dd2650cd-cd7f-44d5-89c2-93c06eb657cb","Type":"ContainerStarted","Data":"f5ad71ea2a1908106559e14bb889d88d1ab90ea818b13eb80c08b361634f49e8"} Jul 10 14:52:04.268729 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:04.268626 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn" Jul 10 14:52:04.301589 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:04.301537 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" podStartSLOduration=1.240994052 podStartE2EDuration="3.301521094s" totalImagesPullingTime="2.060527042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:52:04.299544665 +0000 UTC m=+404.580059362" watchObservedRunningTime="2026-07-10 14:52:04.301521094 +0000 UTC m=+404.582035793" Jul 10 14:52:04.328186 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:04.328134 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn" podStartSLOduration=1.9514736560000001 podStartE2EDuration="5.328117279s" totalImagesPullingTime="3.376643623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:51:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:52:04.327247343 +0000 UTC m=+404.607762040" watchObservedRunningTime="2026-07-10 14:52:04.328117279 +0000 UTC m=+404.608632048" Jul 10 14:52:05.016256 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.016222 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6"] Jul 10 14:52:05.019996 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.019976 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:05.022493 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.022473 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 10 14:52:05.022591 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.022513 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 10 14:52:05.022591 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.022518 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-77sqm\"" Jul 10 14:52:05.030877 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.030856 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6"] Jul 10 14:52:05.142118 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.142083 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gzdq6\" (UID: \"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:05.142275 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.142133 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gzdq6\" (UID: \"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:05.142275 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.142199 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pr5n\" (UniqueName: \"kubernetes.io/projected/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-kube-api-access-6pr5n\") pod \"kuadrant-console-plugin-66b5774985-gzdq6\" (UID: \"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:05.242639 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.242607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pr5n\" (UniqueName: \"kubernetes.io/projected/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-kube-api-access-6pr5n\") pod \"kuadrant-console-plugin-66b5774985-gzdq6\" (UID: \"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:05.242839 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.242669 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gzdq6\" (UID: \"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:05.242839 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.242705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gzdq6\" (UID: \"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:05.242839 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:05.242801 2562 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 10 14:52:05.242995 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:05.242851 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-plugin-serving-cert podName:7fd000f6-c12f-4e15-8141-b7c5d7dcabf8 nodeName:}" failed. No retries permitted until 2026-07-10 14:52:05.742836234 +0000 UTC m=+406.023350909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-gzdq6" (UID: "7fd000f6-c12f-4e15-8141-b7c5d7dcabf8") : secret "plugin-serving-cert" not found Jul 10 14:52:05.243313 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.243292 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gzdq6\" (UID: \"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:05.252758 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.252731 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pr5n\" (UniqueName: \"kubernetes.io/projected/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-kube-api-access-6pr5n\") pod \"kuadrant-console-plugin-66b5774985-gzdq6\" (UID: \"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:05.747676 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.747642 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gzdq6\" (UID: \"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:05.750167 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.750147 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fd000f6-c12f-4e15-8141-b7c5d7dcabf8-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gzdq6\" (UID: \"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:05.929440 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:05.929411 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" Jul 10 14:52:06.079235 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:06.079205 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6"] Jul 10 14:52:06.080946 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:52:06.080924 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fd000f6_c12f_4e15_8141_b7c5d7dcabf8.slice/crio-04904532c4653b98fe04f9813708256b416f25748ebd59d9b8c78196d5916f86 WatchSource:0}: Error finding container 04904532c4653b98fe04f9813708256b416f25748ebd59d9b8c78196d5916f86: Status 404 returned error can't find the container with id 04904532c4653b98fe04f9813708256b416f25748ebd59d9b8c78196d5916f86 Jul 10 14:52:06.281025 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:06.280948 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" event={"ID":"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8","Type":"ContainerStarted","Data":"04904532c4653b98fe04f9813708256b416f25748ebd59d9b8c78196d5916f86"} Jul 10 14:52:12.254862 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:12.254831 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-82csm" Jul 10 14:52:13.261618 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:13.261575 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:15.276410 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.276377 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:15.276858 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.276761 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-49tqn" Jul 10 14:52:15.647320 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.647242 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk"] Jul 10 14:52:15.647577 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.647516 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" podUID="f6227887-18dd-4108-8605-1d3888080933" containerName="manager" containerID="cri-o://8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8" gracePeriod=2 Jul 10 14:52:15.670308 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.670233 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk"] Jul 10 14:52:15.686109 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:15.686060 2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f6227887-18dd-4108-8605-1d3888080933\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\": pods \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:15.692202 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.692173 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk"] Jul 10 14:52:15.692674 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.692620 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" containerName="manager" containerID="cri-o://390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79" gracePeriod=2 Jul 10 14:52:15.697199 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.697175 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk"] Jul 10 14:52:15.697790 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:15.697729 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="f6227887-18dd-4108-8605-1d3888080933" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:15.702273 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:15.702241 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mkmrk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:15.717438 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.717407 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245"] Jul 10 14:52:15.718002 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.717985 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6227887-18dd-4108-8605-1d3888080933" containerName="manager" Jul 10 14:52:15.718087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.718005 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6227887-18dd-4108-8605-1d3888080933" containerName="manager" Jul 10 14:52:15.718087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.718047 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" containerName="manager" Jul 10 14:52:15.718087 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.718055 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" containerName="manager" Jul 10 14:52:15.718205 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.718141 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" containerName="manager" Jul 10 14:52:15.718205 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.718157 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6227887-18dd-4108-8605-1d3888080933" containerName="manager" Jul 10 14:52:15.722776 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.722754 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:15.739179 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.739151 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245"] Jul 10 14:52:15.773243 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:15.773180 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="f6227887-18dd-4108-8605-1d3888080933" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:15.776118 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:15.776085 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mkmrk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:15.797827 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.797797 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf"] Jul 10 14:52:15.804387 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.804357 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf" Jul 10 14:52:15.808528 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:15.808490 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="f6227887-18dd-4108-8605-1d3888080933" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:15.828212 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.828183 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf"] Jul 10 14:52:15.846313 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.846284 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klm86\" (UniqueName: \"kubernetes.io/projected/6502b419-2b1b-41d7-96c0-35049fc54dc3-kube-api-access-klm86\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk245\" (UID: \"6502b419-2b1b-41d7-96c0-35049fc54dc3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:15.846487 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.846369 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6502b419-2b1b-41d7-96c0-35049fc54dc3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk245\" (UID: \"6502b419-2b1b-41d7-96c0-35049fc54dc3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:15.870787 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:15.870740 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mkmrk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:15.947354 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.947256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4d6b\" (UniqueName: \"kubernetes.io/projected/a25fabad-7c56-4d6f-9a20-0efd4f358d8c-kube-api-access-c4d6b\") pod \"limitador-operator-controller-manager-859d7b78bd-smfzf\" (UID: \"a25fabad-7c56-4d6f-9a20-0efd4f358d8c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf" Jul 10 14:52:15.947354 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.947301 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klm86\" (UniqueName: \"kubernetes.io/projected/6502b419-2b1b-41d7-96c0-35049fc54dc3-kube-api-access-klm86\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk245\" (UID: \"6502b419-2b1b-41d7-96c0-35049fc54dc3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:15.947553 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.947364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6502b419-2b1b-41d7-96c0-35049fc54dc3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk245\" (UID: \"6502b419-2b1b-41d7-96c0-35049fc54dc3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:15.947722 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.947698 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6502b419-2b1b-41d7-96c0-35049fc54dc3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk245\" (UID: \"6502b419-2b1b-41d7-96c0-35049fc54dc3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:15.963683 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:15.963651 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klm86\" (UniqueName: \"kubernetes.io/projected/6502b419-2b1b-41d7-96c0-35049fc54dc3-kube-api-access-klm86\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fk245\" (UID: \"6502b419-2b1b-41d7-96c0-35049fc54dc3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:16.048057 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:16.048014 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4d6b\" (UniqueName: \"kubernetes.io/projected/a25fabad-7c56-4d6f-9a20-0efd4f358d8c-kube-api-access-c4d6b\") pod \"limitador-operator-controller-manager-859d7b78bd-smfzf\" (UID: \"a25fabad-7c56-4d6f-9a20-0efd4f358d8c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf" Jul 10 14:52:16.059826 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:16.059766 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4d6b\" (UniqueName: \"kubernetes.io/projected/a25fabad-7c56-4d6f-9a20-0efd4f358d8c-kube-api-access-c4d6b\") pod \"limitador-operator-controller-manager-859d7b78bd-smfzf\" (UID: \"a25fabad-7c56-4d6f-9a20-0efd4f358d8c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf" Jul 10 14:52:16.086164 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:16.086131 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:16.117192 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:16.117157 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf" Jul 10 14:52:28.647870 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.647836 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:28.654560 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:28.654510 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="f6227887-18dd-4108-8605-1d3888080933" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:28.656732 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.656651 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf"] Jul 10 14:52:28.658766 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:52:28.658741 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25fabad_7c56_4d6f_9a20_0efd4f358d8c.slice/crio-6f2250ab1b23fa34ac252f7650f5ed61e7d871e0971d73377b5f93c5835a07f0 WatchSource:0}: Error finding container 6f2250ab1b23fa34ac252f7650f5ed61e7d871e0971d73377b5f93c5835a07f0: Status 404 returned error can't find the container with id 6f2250ab1b23fa34ac252f7650f5ed61e7d871e0971d73377b5f93c5835a07f0 Jul 10 14:52:28.663213 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.663192 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:28.668161 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:28.668128 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="f6227887-18dd-4108-8605-1d3888080933" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:28.671126 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:28.671095 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mkmrk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:28.750459 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.750435 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245"] Jul 10 14:52:28.753091 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:52:28.753060 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6502b419_2b1b_41d7_96c0_35049fc54dc3.slice/crio-8774f873ffc9899a91b01940b8e76c51e28fb97c7b6efa6ce673b1559dda6b66 WatchSource:0}: Error finding container 8774f873ffc9899a91b01940b8e76c51e28fb97c7b6efa6ce673b1559dda6b66: Status 404 returned error can't find the container with id 8774f873ffc9899a91b01940b8e76c51e28fb97c7b6efa6ce673b1559dda6b66 Jul 10 14:52:28.773253 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.773231 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6227887-18dd-4108-8605-1d3888080933-extensions-socket-volume\") pod \"f6227887-18dd-4108-8605-1d3888080933\" (UID: \"f6227887-18dd-4108-8605-1d3888080933\") " Jul 10 14:52:28.773253 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.773263 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7qpd\" (UniqueName: \"kubernetes.io/projected/f6227887-18dd-4108-8605-1d3888080933-kube-api-access-n7qpd\") pod \"f6227887-18dd-4108-8605-1d3888080933\" (UID: \"f6227887-18dd-4108-8605-1d3888080933\") " Jul 10 14:52:28.773463 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.773368 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqjft\" (UniqueName: \"kubernetes.io/projected/8c580aab-8c72-4662-91bf-6accbe55177d-kube-api-access-nqjft\") pod \"8c580aab-8c72-4662-91bf-6accbe55177d\" (UID: \"8c580aab-8c72-4662-91bf-6accbe55177d\") " Jul 10 14:52:28.773840 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.773773 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6227887-18dd-4108-8605-1d3888080933-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f6227887-18dd-4108-8605-1d3888080933" (UID: "f6227887-18dd-4108-8605-1d3888080933"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:52:28.775678 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.775653 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6227887-18dd-4108-8605-1d3888080933-kube-api-access-n7qpd" (OuterVolumeSpecName: "kube-api-access-n7qpd") pod "f6227887-18dd-4108-8605-1d3888080933" (UID: "f6227887-18dd-4108-8605-1d3888080933"). InnerVolumeSpecName "kube-api-access-n7qpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:52:28.776093 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.776057 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c580aab-8c72-4662-91bf-6accbe55177d-kube-api-access-nqjft" (OuterVolumeSpecName: "kube-api-access-nqjft") pod "8c580aab-8c72-4662-91bf-6accbe55177d" (UID: "8c580aab-8c72-4662-91bf-6accbe55177d"). InnerVolumeSpecName "kube-api-access-nqjft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:52:28.874105 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.874074 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6227887-18dd-4108-8605-1d3888080933-extensions-socket-volume\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:52:28.874105 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.874101 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7qpd\" (UniqueName: \"kubernetes.io/projected/f6227887-18dd-4108-8605-1d3888080933-kube-api-access-n7qpd\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:52:28.874105 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:28.874110 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqjft\" (UniqueName: \"kubernetes.io/projected/8c580aab-8c72-4662-91bf-6accbe55177d-kube-api-access-nqjft\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:52:29.438408 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.438354 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf" event={"ID":"a25fabad-7c56-4d6f-9a20-0efd4f358d8c","Type":"ContainerStarted","Data":"deb3fe28adb8a390e68580271b3f44d42b44ac8cb5df6843f5a4b8dca1c444bd"} Jul 10 14:52:29.438584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.438434 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf" Jul 10 14:52:29.438584 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.438461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf" event={"ID":"a25fabad-7c56-4d6f-9a20-0efd4f358d8c","Type":"ContainerStarted","Data":"6f2250ab1b23fa34ac252f7650f5ed61e7d871e0971d73377b5f93c5835a07f0"} Jul 10 14:52:29.439878 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.439852 2562 generic.go:356] "Generic (PLEG): container finished" podID="f6227887-18dd-4108-8605-1d3888080933" containerID="8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8" exitCode=0 Jul 10 14:52:29.439979 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.439895 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:29.439979 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.439929 2562 scope.go:117] "RemoveContainer" containerID="8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8" Jul 10 14:52:29.441751 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.441713 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" event={"ID":"7fd000f6-c12f-4e15-8141-b7c5d7dcabf8","Type":"ContainerStarted","Data":"e3c05b35e14c10520e499de4ae83ae005006b6927070ebbb3755d2ff3a069f6e"} Jul 10 14:52:29.443500 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.443477 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" event={"ID":"6502b419-2b1b-41d7-96c0-35049fc54dc3","Type":"ContainerStarted","Data":"c4b4513d30a0745a9294fdf8f557641669687e922d116abe6d871bffe536d690"} Jul 10 14:52:29.443604 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.443519 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" event={"ID":"6502b419-2b1b-41d7-96c0-35049fc54dc3","Type":"ContainerStarted","Data":"8774f873ffc9899a91b01940b8e76c51e28fb97c7b6efa6ce673b1559dda6b66"} Jul 10 14:52:29.443604 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.443598 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:29.444966 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.444944 2562 generic.go:356] "Generic (PLEG): container finished" podID="8c580aab-8c72-4662-91bf-6accbe55177d" containerID="390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79" exitCode=0 Jul 10 14:52:29.445053 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.444983 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:29.450794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.450777 2562 scope.go:117] "RemoveContainer" containerID="8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8" Jul 10 14:52:29.451116 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:29.451090 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8\": container with ID starting with 8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8 not found: ID does not exist" containerID="8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8" Jul 10 14:52:29.451197 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.451128 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8"} err="failed to get container status \"8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8\": rpc error: code = NotFound desc = could not find container \"8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8\": container with ID starting with 8faf34f0a2ccaad87958ebca07e4a7b66b3ced419a1889a689a4b259a7b9dbb8 not found: ID does not exist" Jul 10 14:52:29.451197 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.451152 2562 scope.go:117] "RemoveContainer" containerID="390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79" Jul 10 14:52:29.460148 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.460126 2562 scope.go:117] "RemoveContainer" containerID="390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79" Jul 10 14:52:29.460524 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:29.460498 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79\": container with ID starting with 390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79 not found: ID does not exist" containerID="390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79" Jul 10 14:52:29.460680 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.460655 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79"} err="failed to get container status \"390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79\": rpc error: code = NotFound desc = could not find container \"390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79\": container with ID starting with 390eda625d933859ede6a5791256200b69b1d2b955a0529d8d72e763f191ec79 not found: ID does not exist" Jul 10 14:52:29.473243 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:29.473130 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mkmrk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:29.474545 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.474496 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf" podStartSLOduration=14.474477968 podStartE2EDuration="14.474477968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:52:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:52:29.470849267 +0000 UTC m=+429.751363959" watchObservedRunningTime="2026-07-10 14:52:29.474477968 +0000 UTC m=+429.754992667" Jul 10 14:52:29.477312 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:29.477250 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="f6227887-18dd-4108-8605-1d3888080933" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:29.541682 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.541629 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" podStartSLOduration=14.541612238999999 podStartE2EDuration="14.541612239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:52:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:52:29.539831322 +0000 UTC m=+429.820346021" watchObservedRunningTime="2026-07-10 14:52:29.541612239 +0000 UTC m=+429.822126914" Jul 10 14:52:29.567169 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:29.567116 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gzdq6" podStartSLOduration=3.079338279 podStartE2EDuration="25.567094776s" totalImagesPullingTime="22.487756497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:52:29.565554116 +0000 UTC m=+429.846068815" watchObservedRunningTime="2026-07-10 14:52:29.567094776 +0000 UTC m=+429.847609474" Jul 10 14:52:29.570442 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:29.570410 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="f6227887-18dd-4108-8605-1d3888080933" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:29.572463 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:29.572444 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mkmrk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:30.279524 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:30.279483 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jvrhk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="f6227887-18dd-4108-8605-1d3888080933" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jvrhk" Jul 10 14:52:30.279966 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:30.279929 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" path="/var/lib/kubelet/pods/8c580aab-8c72-4662-91bf-6accbe55177d/volumes" Jul 10 14:52:30.280427 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:30.280409 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6227887-18dd-4108-8605-1d3888080933" path="/var/lib/kubelet/pods/f6227887-18dd-4108-8605-1d3888080933/volumes" Jul 10 14:52:30.281292 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:52:30.281268 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mkmrk\" is forbidden: User \"system:node:ip-10-0-135-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-1.ec2.internal' and this object" podUID="8c580aab-8c72-4662-91bf-6accbe55177d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mkmrk" Jul 10 14:52:34.807460 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:34.807421 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245"] Jul 10 14:52:34.807902 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:34.807658 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" podUID="6502b419-2b1b-41d7-96c0-35049fc54dc3" containerName="manager" containerID="cri-o://c4b4513d30a0745a9294fdf8f557641669687e922d116abe6d871bffe536d690" gracePeriod=10 Jul 10 14:52:34.809541 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:34.809519 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:35.491270 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:35.491235 2562 generic.go:356] "Generic (PLEG): container finished" podID="6502b419-2b1b-41d7-96c0-35049fc54dc3" containerID="c4b4513d30a0745a9294fdf8f557641669687e922d116abe6d871bffe536d690" exitCode=0 Jul 10 14:52:35.491452 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:35.491373 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" event={"ID":"6502b419-2b1b-41d7-96c0-35049fc54dc3","Type":"ContainerDied","Data":"c4b4513d30a0745a9294fdf8f557641669687e922d116abe6d871bffe536d690"} Jul 10 14:52:35.601095 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:35.601065 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:35.629589 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:35.629560 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klm86\" (UniqueName: \"kubernetes.io/projected/6502b419-2b1b-41d7-96c0-35049fc54dc3-kube-api-access-klm86\") pod \"6502b419-2b1b-41d7-96c0-35049fc54dc3\" (UID: \"6502b419-2b1b-41d7-96c0-35049fc54dc3\") " Jul 10 14:52:35.629732 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:35.629608 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6502b419-2b1b-41d7-96c0-35049fc54dc3-extensions-socket-volume\") pod \"6502b419-2b1b-41d7-96c0-35049fc54dc3\" (UID: \"6502b419-2b1b-41d7-96c0-35049fc54dc3\") " Jul 10 14:52:35.630068 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:35.630037 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6502b419-2b1b-41d7-96c0-35049fc54dc3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6502b419-2b1b-41d7-96c0-35049fc54dc3" (UID: "6502b419-2b1b-41d7-96c0-35049fc54dc3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 14:52:35.632389 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:35.632363 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6502b419-2b1b-41d7-96c0-35049fc54dc3-kube-api-access-klm86" (OuterVolumeSpecName: "kube-api-access-klm86") pod "6502b419-2b1b-41d7-96c0-35049fc54dc3" (UID: "6502b419-2b1b-41d7-96c0-35049fc54dc3"). InnerVolumeSpecName "kube-api-access-klm86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:52:35.730572 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:35.730494 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6502b419-2b1b-41d7-96c0-35049fc54dc3-extensions-socket-volume\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:52:35.730572 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:35.730525 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klm86\" (UniqueName: \"kubernetes.io/projected/6502b419-2b1b-41d7-96c0-35049fc54dc3-kube-api-access-klm86\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:52:36.499024 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:36.498997 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" Jul 10 14:52:36.499456 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:36.498994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245" event={"ID":"6502b419-2b1b-41d7-96c0-35049fc54dc3","Type":"ContainerDied","Data":"8774f873ffc9899a91b01940b8e76c51e28fb97c7b6efa6ce673b1559dda6b66"} Jul 10 14:52:36.499456 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:36.499137 2562 scope.go:117] "RemoveContainer" containerID="c4b4513d30a0745a9294fdf8f557641669687e922d116abe6d871bffe536d690" Jul 10 14:52:36.540595 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:36.540532 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245"] Jul 10 14:52:36.572695 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:36.572664 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fk245"] Jul 10 14:52:38.279248 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:38.279211 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6502b419-2b1b-41d7-96c0-35049fc54dc3" path="/var/lib/kubelet/pods/6502b419-2b1b-41d7-96c0-35049fc54dc3/volumes" Jul 10 14:52:40.455373 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:40.455281 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-smfzf" Jul 10 14:52:54.378282 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.378240 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rblj2"] Jul 10 14:52:54.378820 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.378799 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6502b419-2b1b-41d7-96c0-35049fc54dc3" containerName="manager" Jul 10 14:52:54.378900 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.378823 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6502b419-2b1b-41d7-96c0-35049fc54dc3" containerName="manager" Jul 10 14:52:54.378968 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.378955 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6502b419-2b1b-41d7-96c0-35049fc54dc3" containerName="manager" Jul 10 14:52:54.435468 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.435427 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rblj2"] Jul 10 14:52:54.435632 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.435551 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:52:54.439863 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.439843 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 14:52:54.479121 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.479086 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88g7f\" (UniqueName: \"kubernetes.io/projected/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-kube-api-access-88g7f\") pod \"limitador-limitador-64c94fd64f-rblj2\" (UID: \"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:52:54.479288 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.479156 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-config-file\") pod \"limitador-limitador-64c94fd64f-rblj2\" (UID: \"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:52:54.491716 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.491682 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rblj2"] Jul 10 14:52:54.580083 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.580046 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88g7f\" (UniqueName: \"kubernetes.io/projected/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-kube-api-access-88g7f\") pod \"limitador-limitador-64c94fd64f-rblj2\" (UID: \"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:52:54.580255 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.580106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-config-file\") pod \"limitador-limitador-64c94fd64f-rblj2\" (UID: \"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:52:54.580694 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.580675 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-config-file\") pod \"limitador-limitador-64c94fd64f-rblj2\" (UID: \"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:52:54.596775 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.596753 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88g7f\" (UniqueName: \"kubernetes.io/projected/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-kube-api-access-88g7f\") pod \"limitador-limitador-64c94fd64f-rblj2\" (UID: \"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:52:54.746230 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.746199 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:52:54.878967 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.878934 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:52:54.881404 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.881380 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" Jul 10 14:52:54.901694 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.901663 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:52:54.903782 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:52:54.903755 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7288e14c_4d2f_4a07_9e20_0e3a5f6e48b8.slice/crio-ef0a266148d3cfda6ec98512c5b3faf778065ce432bbe1fd8abefd33b486be50 WatchSource:0}: Error finding container ef0a266148d3cfda6ec98512c5b3faf778065ce432bbe1fd8abefd33b486be50: Status 404 returned error can't find the container with id ef0a266148d3cfda6ec98512c5b3faf778065ce432bbe1fd8abefd33b486be50 Jul 10 14:52:54.904143 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.904124 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rblj2"] Jul 10 14:52:54.924971 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.924945 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:52:54.982108 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.982069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bzw7\" (UniqueName: \"kubernetes.io/projected/361be177-00db-4509-9ccf-9021f36f06cf-kube-api-access-4bzw7\") pod \"limitador-limitador-54d6d988d5-mjnkv\" (UID: \"361be177-00db-4509-9ccf-9021f36f06cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" Jul 10 14:52:54.982108 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:54.982108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/361be177-00db-4509-9ccf-9021f36f06cf-config-file\") pod \"limitador-limitador-54d6d988d5-mjnkv\" (UID: \"361be177-00db-4509-9ccf-9021f36f06cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" Jul 10 14:52:55.082850 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.082744 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bzw7\" (UniqueName: \"kubernetes.io/projected/361be177-00db-4509-9ccf-9021f36f06cf-kube-api-access-4bzw7\") pod \"limitador-limitador-54d6d988d5-mjnkv\" (UID: \"361be177-00db-4509-9ccf-9021f36f06cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" Jul 10 14:52:55.082850 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.082787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/361be177-00db-4509-9ccf-9021f36f06cf-config-file\") pod \"limitador-limitador-54d6d988d5-mjnkv\" (UID: \"361be177-00db-4509-9ccf-9021f36f06cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" Jul 10 14:52:55.083491 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.083465 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/361be177-00db-4509-9ccf-9021f36f06cf-config-file\") pod \"limitador-limitador-54d6d988d5-mjnkv\" (UID: \"361be177-00db-4509-9ccf-9021f36f06cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" Jul 10 14:52:55.092528 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.092501 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bzw7\" (UniqueName: \"kubernetes.io/projected/361be177-00db-4509-9ccf-9021f36f06cf-kube-api-access-4bzw7\") pod \"limitador-limitador-54d6d988d5-mjnkv\" (UID: \"361be177-00db-4509-9ccf-9021f36f06cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" Jul 10 14:52:55.191963 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.191925 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" Jul 10 14:52:55.196763 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.196738 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-4kzgn"] Jul 10 14:52:55.198948 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.198933 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4kzgn" Jul 10 14:52:55.201235 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.201219 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mwm6l\"" Jul 10 14:52:55.217357 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.217316 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4kzgn"] Jul 10 14:52:55.284002 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.283969 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6m4h\" (UniqueName: \"kubernetes.io/projected/45f70d54-8ea3-406a-92ba-0885396186f2-kube-api-access-m6m4h\") pod \"authorino-58cd955675-4kzgn\" (UID: \"45f70d54-8ea3-406a-92ba-0885396186f2\") " pod="kuadrant-system/authorino-58cd955675-4kzgn" Jul 10 14:52:55.335166 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.335141 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:52:55.337516 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:52:55.337489 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod361be177_00db_4509_9ccf_9021f36f06cf.slice/crio-83fe0ada0becc8ab99f9599966a283d36db851fd4301d342a1e48da2b59cae76 WatchSource:0}: Error finding container 83fe0ada0becc8ab99f9599966a283d36db851fd4301d342a1e48da2b59cae76: Status 404 returned error can't find the container with id 83fe0ada0becc8ab99f9599966a283d36db851fd4301d342a1e48da2b59cae76 Jul 10 14:52:55.384540 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.384508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6m4h\" (UniqueName: \"kubernetes.io/projected/45f70d54-8ea3-406a-92ba-0885396186f2-kube-api-access-m6m4h\") pod \"authorino-58cd955675-4kzgn\" (UID: \"45f70d54-8ea3-406a-92ba-0885396186f2\") " pod="kuadrant-system/authorino-58cd955675-4kzgn" Jul 10 14:52:55.395670 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.395646 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6m4h\" (UniqueName: \"kubernetes.io/projected/45f70d54-8ea3-406a-92ba-0885396186f2-kube-api-access-m6m4h\") pod \"authorino-58cd955675-4kzgn\" (UID: \"45f70d54-8ea3-406a-92ba-0885396186f2\") " pod="kuadrant-system/authorino-58cd955675-4kzgn" Jul 10 14:52:55.504562 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.504528 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-zjlzh"] Jul 10 14:52:55.506804 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.506780 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zjlzh" Jul 10 14:52:55.508987 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.508963 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4kzgn" Jul 10 14:52:55.523618 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.523562 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zjlzh"] Jul 10 14:52:55.586301 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.586208 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q646k\" (UniqueName: \"kubernetes.io/projected/6534cd6b-26dc-4537-8417-63ed7a7ac19b-kube-api-access-q646k\") pod \"authorino-75d556bd77-zjlzh\" (UID: \"6534cd6b-26dc-4537-8417-63ed7a7ac19b\") " pod="kuadrant-system/authorino-75d556bd77-zjlzh" Jul 10 14:52:55.613448 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.613319 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" event={"ID":"361be177-00db-4509-9ccf-9021f36f06cf","Type":"ContainerStarted","Data":"83fe0ada0becc8ab99f9599966a283d36db851fd4301d342a1e48da2b59cae76"} Jul 10 14:52:55.615513 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.615449 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" event={"ID":"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8","Type":"ContainerStarted","Data":"ef0a266148d3cfda6ec98512c5b3faf778065ce432bbe1fd8abefd33b486be50"} Jul 10 14:52:55.687417 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.687274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q646k\" (UniqueName: \"kubernetes.io/projected/6534cd6b-26dc-4537-8417-63ed7a7ac19b-kube-api-access-q646k\") pod \"authorino-75d556bd77-zjlzh\" (UID: \"6534cd6b-26dc-4537-8417-63ed7a7ac19b\") " pod="kuadrant-system/authorino-75d556bd77-zjlzh" Jul 10 14:52:55.689010 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.688722 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4kzgn"] Jul 10 14:52:55.690840 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:52:55.690810 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45f70d54_8ea3_406a_92ba_0885396186f2.slice/crio-f2be17d02bb9fd2be35e83376a1af9de2fb9c925db4d83e8780ac5fcac6d8a58 WatchSource:0}: Error finding container f2be17d02bb9fd2be35e83376a1af9de2fb9c925db4d83e8780ac5fcac6d8a58: Status 404 returned error can't find the container with id f2be17d02bb9fd2be35e83376a1af9de2fb9c925db4d83e8780ac5fcac6d8a58 Jul 10 14:52:55.707502 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.707476 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q646k\" (UniqueName: \"kubernetes.io/projected/6534cd6b-26dc-4537-8417-63ed7a7ac19b-kube-api-access-q646k\") pod \"authorino-75d556bd77-zjlzh\" (UID: \"6534cd6b-26dc-4537-8417-63ed7a7ac19b\") " pod="kuadrant-system/authorino-75d556bd77-zjlzh" Jul 10 14:52:55.816201 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:55.816169 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zjlzh" Jul 10 14:52:56.041651 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:56.041617 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zjlzh"] Jul 10 14:52:56.049043 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:52:56.049008 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6534cd6b_26dc_4537_8417_63ed7a7ac19b.slice/crio-b3771309e54ae584c25322dd4b0f36e7651f00866aaae7078a66fef78bfab0db WatchSource:0}: Error finding container b3771309e54ae584c25322dd4b0f36e7651f00866aaae7078a66fef78bfab0db: Status 404 returned error can't find the container with id b3771309e54ae584c25322dd4b0f36e7651f00866aaae7078a66fef78bfab0db Jul 10 14:52:56.623607 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:56.623540 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zjlzh" event={"ID":"6534cd6b-26dc-4537-8417-63ed7a7ac19b","Type":"ContainerStarted","Data":"b3771309e54ae584c25322dd4b0f36e7651f00866aaae7078a66fef78bfab0db"} Jul 10 14:52:56.624958 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:52:56.624932 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4kzgn" event={"ID":"45f70d54-8ea3-406a-92ba-0885396186f2","Type":"ContainerStarted","Data":"f2be17d02bb9fd2be35e83376a1af9de2fb9c925db4d83e8780ac5fcac6d8a58"} Jul 10 14:53:00.654048 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:00.654013 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" event={"ID":"361be177-00db-4509-9ccf-9021f36f06cf","Type":"ContainerStarted","Data":"4122c25a436a82e03295320cc3efc02ab26862f85f1d862bc3c36693a12a24ad"} Jul 10 14:53:00.654560 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:00.654134 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" Jul 10 14:53:00.655728 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:00.655702 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" event={"ID":"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8","Type":"ContainerStarted","Data":"eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142"} Jul 10 14:53:00.655841 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:00.655786 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:53:00.657200 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:00.657181 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zjlzh" event={"ID":"6534cd6b-26dc-4537-8417-63ed7a7ac19b","Type":"ContainerStarted","Data":"b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2"} Jul 10 14:53:00.658660 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:00.658635 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4kzgn" event={"ID":"45f70d54-8ea3-406a-92ba-0885396186f2","Type":"ContainerStarted","Data":"2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774"} Jul 10 14:53:00.681686 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:00.681641 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" podStartSLOduration=1.9208058590000001 podStartE2EDuration="6.681625961s" totalImagesPullingTime="4.760820102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:52:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:53:00.679708307 +0000 UTC m=+460.960222987" watchObservedRunningTime="2026-07-10 14:53:00.681625961 +0000 UTC m=+460.962140658" Jul 10 14:53:00.700066 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:00.700018 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-zjlzh" podStartSLOduration=1.580112684 podStartE2EDuration="5.7000022s" totalImagesPullingTime="4.119889516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:53:00.699627719 +0000 UTC m=+460.980142419" watchObservedRunningTime="2026-07-10 14:53:00.7000022 +0000 UTC m=+460.980516876" Jul 10 14:53:00.724529 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:00.724481 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" podStartSLOduration=1.529814939 podStartE2EDuration="6.724464038s" totalImagesPullingTime="5.194649099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:52:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:53:00.723421308 +0000 UTC m=+461.003936008" watchObservedRunningTime="2026-07-10 14:53:00.724464038 +0000 UTC m=+461.004978736" Jul 10 14:53:00.754860 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:00.754820 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4kzgn"] Jul 10 14:53:00.770566 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:00.770512 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-4kzgn" podStartSLOduration=1.279039171 podStartE2EDuration="5.770495106s" totalImagesPullingTime="4.491455935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:53:00.768350517 +0000 UTC m=+461.048865216" watchObservedRunningTime="2026-07-10 14:53:00.770495106 +0000 UTC m=+461.051009805" Jul 10 14:53:02.670010 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:02.669966 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-4kzgn" podUID="45f70d54-8ea3-406a-92ba-0885396186f2" containerName="authorino" containerID="cri-o://2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774" gracePeriod=30 Jul 10 14:53:02.928190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:02.928114 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4kzgn" Jul 10 14:53:02.958140 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:02.958106 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6m4h\" (UniqueName: \"kubernetes.io/projected/45f70d54-8ea3-406a-92ba-0885396186f2-kube-api-access-m6m4h\") pod \"45f70d54-8ea3-406a-92ba-0885396186f2\" (UID: \"45f70d54-8ea3-406a-92ba-0885396186f2\") " Jul 10 14:53:02.960594 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:02.960560 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45f70d54-8ea3-406a-92ba-0885396186f2-kube-api-access-m6m4h" (OuterVolumeSpecName: "kube-api-access-m6m4h") pod "45f70d54-8ea3-406a-92ba-0885396186f2" (UID: "45f70d54-8ea3-406a-92ba-0885396186f2"). InnerVolumeSpecName "kube-api-access-m6m4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:53:03.059202 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:03.059163 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6m4h\" (UniqueName: \"kubernetes.io/projected/45f70d54-8ea3-406a-92ba-0885396186f2-kube-api-access-m6m4h\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:53:03.677291 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:03.677252 2562 generic.go:356] "Generic (PLEG): container finished" podID="45f70d54-8ea3-406a-92ba-0885396186f2" containerID="2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774" exitCode=0 Jul 10 14:53:03.677747 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:03.677300 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4kzgn" Jul 10 14:53:03.677747 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:03.677349 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4kzgn" event={"ID":"45f70d54-8ea3-406a-92ba-0885396186f2","Type":"ContainerDied","Data":"2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774"} Jul 10 14:53:03.677747 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:03.677392 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4kzgn" event={"ID":"45f70d54-8ea3-406a-92ba-0885396186f2","Type":"ContainerDied","Data":"f2be17d02bb9fd2be35e83376a1af9de2fb9c925db4d83e8780ac5fcac6d8a58"} Jul 10 14:53:03.677747 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:03.677416 2562 scope.go:117] "RemoveContainer" containerID="2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774" Jul 10 14:53:03.687136 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:03.687115 2562 scope.go:117] "RemoveContainer" containerID="2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774" Jul 10 14:53:03.687511 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:53:03.687491 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774\": container with ID starting with 2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774 not found: ID does not exist" containerID="2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774" Jul 10 14:53:03.687575 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:03.687519 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774"} err="failed to get container status \"2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774\": rpc error: code = NotFound desc = could not find container \"2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774\": container with ID starting with 2d66c86344b70c94453ec116710b4af0f30b72884e314b4af050bd272cbea774 not found: ID does not exist" Jul 10 14:53:03.710111 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:03.710074 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4kzgn"] Jul 10 14:53:03.713918 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:03.713895 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-4kzgn"] Jul 10 14:53:04.280439 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:04.280409 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45f70d54-8ea3-406a-92ba-0885396186f2" path="/var/lib/kubelet/pods/45f70d54-8ea3-406a-92ba-0885396186f2/volumes" Jul 10 14:53:11.665556 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:11.665521 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-mjnkv" Jul 10 14:53:11.666911 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:11.666889 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:53:11.755374 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:11.755327 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rblj2"] Jul 10 14:53:11.755550 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:11.755528 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" podUID="7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8" containerName="limitador" containerID="cri-o://eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142" gracePeriod=30 Jul 10 14:53:12.315999 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.315936 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:53:12.445061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.445029 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-config-file\") pod \"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8\" (UID: \"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8\") " Jul 10 14:53:12.445245 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.445127 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88g7f\" (UniqueName: \"kubernetes.io/projected/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-kube-api-access-88g7f\") pod \"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8\" (UID: \"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8\") " Jul 10 14:53:12.445468 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.445446 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-config-file" (OuterVolumeSpecName: "config-file") pod "7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8" (UID: "7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 14:53:12.447290 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.447271 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-kube-api-access-88g7f" (OuterVolumeSpecName: "kube-api-access-88g7f") pod "7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8" (UID: "7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8"). InnerVolumeSpecName "kube-api-access-88g7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:53:12.545668 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.545626 2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-config-file\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:53:12.545668 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.545658 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88g7f\" (UniqueName: \"kubernetes.io/projected/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8-kube-api-access-88g7f\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:53:12.729503 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.729414 2562 generic.go:356] "Generic (PLEG): container finished" podID="7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8" containerID="eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142" exitCode=0 Jul 10 14:53:12.729503 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.729465 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" Jul 10 14:53:12.729979 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.729500 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" event={"ID":"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8","Type":"ContainerDied","Data":"eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142"} Jul 10 14:53:12.729979 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.729552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rblj2" event={"ID":"7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8","Type":"ContainerDied","Data":"ef0a266148d3cfda6ec98512c5b3faf778065ce432bbe1fd8abefd33b486be50"} Jul 10 14:53:12.729979 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.729566 2562 scope.go:117] "RemoveContainer" containerID="eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142" Jul 10 14:53:12.739511 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.739495 2562 scope.go:117] "RemoveContainer" containerID="eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142" Jul 10 14:53:12.739758 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:53:12.739740 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142\": container with ID starting with eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142 not found: ID does not exist" containerID="eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142" Jul 10 14:53:12.739823 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.739767 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142"} err="failed to get container status \"eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142\": rpc error: code = NotFound desc = could not find container \"eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142\": container with ID starting with eef835671e1bedd2b6f99231bd853e49d0ee59c4dfe54b89c6d2b3cc0c450142 not found: ID does not exist" Jul 10 14:53:12.754598 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.754574 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rblj2"] Jul 10 14:53:12.758807 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:12.758786 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rblj2"] Jul 10 14:53:14.280139 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:14.280107 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8" path="/var/lib/kubelet/pods/7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8/volumes" Jul 10 14:53:15.911933 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.911897 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh"] Jul 10 14:53:15.912301 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.912287 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8" containerName="limitador" Jul 10 14:53:15.912428 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.912303 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8" containerName="limitador" Jul 10 14:53:15.912428 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.912317 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45f70d54-8ea3-406a-92ba-0885396186f2" containerName="authorino" Jul 10 14:53:15.912428 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.912322 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="45f70d54-8ea3-406a-92ba-0885396186f2" containerName="authorino" Jul 10 14:53:15.912428 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.912419 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="45f70d54-8ea3-406a-92ba-0885396186f2" containerName="authorino" Jul 10 14:53:15.912556 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.912431 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7288e14c-4d2f-4a07-9e20-0e3a5f6e48b8" containerName="limitador" Jul 10 14:53:15.916816 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.916795 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" Jul 10 14:53:15.919577 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.919557 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zr566\"" Jul 10 14:53:15.919986 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.919967 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 14:53:15.920476 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.920463 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 14:53:15.920530 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.920471 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 10 14:53:15.940963 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.940927 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh"] Jul 10 14:53:15.975273 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.975237 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/437af4cc-a5d3-4092-b36e-818ccceae961-data\") pod \"postgres-69fb4594cf-x7mdh\" (UID: \"437af4cc-a5d3-4092-b36e-818ccceae961\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" Jul 10 14:53:15.975273 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:15.975277 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dbmg\" (UniqueName: \"kubernetes.io/projected/437af4cc-a5d3-4092-b36e-818ccceae961-kube-api-access-5dbmg\") pod \"postgres-69fb4594cf-x7mdh\" (UID: \"437af4cc-a5d3-4092-b36e-818ccceae961\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" Jul 10 14:53:16.076779 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:16.076751 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/437af4cc-a5d3-4092-b36e-818ccceae961-data\") pod \"postgres-69fb4594cf-x7mdh\" (UID: \"437af4cc-a5d3-4092-b36e-818ccceae961\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" Jul 10 14:53:16.076779 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:16.076781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dbmg\" (UniqueName: \"kubernetes.io/projected/437af4cc-a5d3-4092-b36e-818ccceae961-kube-api-access-5dbmg\") pod \"postgres-69fb4594cf-x7mdh\" (UID: \"437af4cc-a5d3-4092-b36e-818ccceae961\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" Jul 10 14:53:16.077109 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:16.077091 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/437af4cc-a5d3-4092-b36e-818ccceae961-data\") pod \"postgres-69fb4594cf-x7mdh\" (UID: \"437af4cc-a5d3-4092-b36e-818ccceae961\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" Jul 10 14:53:16.084928 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:16.084906 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dbmg\" (UniqueName: \"kubernetes.io/projected/437af4cc-a5d3-4092-b36e-818ccceae961-kube-api-access-5dbmg\") pod \"postgres-69fb4594cf-x7mdh\" (UID: \"437af4cc-a5d3-4092-b36e-818ccceae961\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" Jul 10 14:53:16.227177 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:16.227088 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" Jul 10 14:53:16.368691 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:16.368667 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh"] Jul 10 14:53:16.370730 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:53:16.370699 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod437af4cc_a5d3_4092_b36e_818ccceae961.slice/crio-abfe3a6facf2b93720c7735c065d342ea658ce5105fd2af12483a7c066565c0c WatchSource:0}: Error finding container abfe3a6facf2b93720c7735c065d342ea658ce5105fd2af12483a7c066565c0c: Status 404 returned error can't find the container with id abfe3a6facf2b93720c7735c065d342ea658ce5105fd2af12483a7c066565c0c Jul 10 14:53:16.754742 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:16.754704 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" event={"ID":"437af4cc-a5d3-4092-b36e-818ccceae961","Type":"ContainerStarted","Data":"abfe3a6facf2b93720c7735c065d342ea658ce5105fd2af12483a7c066565c0c"} Jul 10 14:53:21.405408 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:21.405384 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 10 14:53:21.790406 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:21.790365 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" event={"ID":"437af4cc-a5d3-4092-b36e-818ccceae961","Type":"ContainerStarted","Data":"789c2cc76c16da4cb51f7cfcfa0d8081eace0cd0212591f59a35198601b0c2e9"} Jul 10 14:53:21.790603 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:21.790435 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" Jul 10 14:53:21.813325 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:21.813278 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" podStartSLOduration=1.7836995679999998 podStartE2EDuration="6.813264075s" totalImagesPullingTime="5.029564507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:53:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:53:21.811390257 +0000 UTC m=+482.091904956" watchObservedRunningTime="2026-07-10 14:53:21.813264075 +0000 UTC m=+482.093778812" Jul 10 14:53:27.826662 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:27.826633 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x7mdh" Jul 10 14:53:28.413946 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.413908 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vx5fh"] Jul 10 14:53:28.420607 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.420582 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vx5fh" Jul 10 14:53:28.427055 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.427032 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vx5fh"] Jul 10 14:53:28.474414 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.474376 2562 scope.go:117] "RemoveContainer" containerID="b83f9ca1ed0dda08fc96f3528168da904c7c77ee5bd583f57d5abacc2bbab270" Jul 10 14:53:28.483306 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.483287 2562 scope.go:117] "RemoveContainer" containerID="ce8d4921432acd06b79da7aa1ef5bbc91dc6c95a0297993469c977f5d54cf63e" Jul 10 14:53:28.491968 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.491949 2562 scope.go:117] "RemoveContainer" containerID="a5b10b44803de767667fb24eb7c40b35ea6c56d4665161b780a5a0db5358b660" Jul 10 14:53:28.499732 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.499715 2562 scope.go:117] "RemoveContainer" containerID="1cbf58dff4df9092338b7be800511b3fa7b6f37b0f8245cf6e5a5ec45c390d0c" Jul 10 14:53:28.507194 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.507176 2562 scope.go:117] "RemoveContainer" containerID="08dd84d3aff4da2a9c797ba33970254d959a0126ba1295d7418de28bbedb1dc5" Jul 10 14:53:28.514769 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.514750 2562 scope.go:117] "RemoveContainer" containerID="61ca9d40a4e33be94377014296c07aaa1c5ba69688f419832a1d262e13b0d20c" Jul 10 14:53:28.522274 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.522252 2562 scope.go:117] "RemoveContainer" containerID="57d95a4a63143e0c2285ba0509b1c602190a13a3d4e8911a40d422f5a01329a8" Jul 10 14:53:28.589572 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.589540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6drfg\" (UniqueName: \"kubernetes.io/projected/667ef294-7065-410a-81fe-b374a36f4892-kube-api-access-6drfg\") pod \"authorino-5d4b67c446-vx5fh\" (UID: \"667ef294-7065-410a-81fe-b374a36f4892\") " pod="kuadrant-system/authorino-5d4b67c446-vx5fh" Jul 10 14:53:28.671794 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.671714 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vx5fh"] Jul 10 14:53:28.671994 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:53:28.671975 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-6drfg], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-vx5fh" podUID="667ef294-7065-410a-81fe-b374a36f4892" Jul 10 14:53:28.690590 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.690564 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6drfg\" (UniqueName: \"kubernetes.io/projected/667ef294-7065-410a-81fe-b374a36f4892-kube-api-access-6drfg\") pod \"authorino-5d4b67c446-vx5fh\" (UID: \"667ef294-7065-410a-81fe-b374a36f4892\") " pod="kuadrant-system/authorino-5d4b67c446-vx5fh" Jul 10 14:53:28.710049 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.710018 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9bc8d787b-pg5hw"] Jul 10 14:53:28.712189 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.712172 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6drfg\" (UniqueName: \"kubernetes.io/projected/667ef294-7065-410a-81fe-b374a36f4892-kube-api-access-6drfg\") pod \"authorino-5d4b67c446-vx5fh\" (UID: \"667ef294-7065-410a-81fe-b374a36f4892\") " pod="kuadrant-system/authorino-5d4b67c446-vx5fh" Jul 10 14:53:28.722705 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.722682 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bc8d787b-pg5hw" Jul 10 14:53:28.722976 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.722956 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9bc8d787b-pg5hw"] Jul 10 14:53:28.837006 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.836978 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vx5fh" Jul 10 14:53:28.844265 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.844246 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vx5fh" Jul 10 14:53:28.884201 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.884170 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9bc8d787b-pg5hw"] Jul 10 14:53:28.884453 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:53:28.884435 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-bz5hs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-9bc8d787b-pg5hw" podUID="2e83c0a9-9f0b-4268-8369-d49bc08b8d32" Jul 10 14:53:28.892118 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.892097 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz5hs\" (UniqueName: \"kubernetes.io/projected/2e83c0a9-9f0b-4268-8369-d49bc08b8d32-kube-api-access-bz5hs\") pod \"authorino-9bc8d787b-pg5hw\" (UID: \"2e83c0a9-9f0b-4268-8369-d49bc08b8d32\") " pod="kuadrant-system/authorino-9bc8d787b-pg5hw" Jul 10 14:53:28.993300 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.993208 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6drfg\" (UniqueName: \"kubernetes.io/projected/667ef294-7065-410a-81fe-b374a36f4892-kube-api-access-6drfg\") pod \"667ef294-7065-410a-81fe-b374a36f4892\" (UID: \"667ef294-7065-410a-81fe-b374a36f4892\") " Jul 10 14:53:28.993498 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.993368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz5hs\" (UniqueName: \"kubernetes.io/projected/2e83c0a9-9f0b-4268-8369-d49bc08b8d32-kube-api-access-bz5hs\") pod \"authorino-9bc8d787b-pg5hw\" (UID: \"2e83c0a9-9f0b-4268-8369-d49bc08b8d32\") " pod="kuadrant-system/authorino-9bc8d787b-pg5hw" Jul 10 14:53:28.995466 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:28.995441 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/667ef294-7065-410a-81fe-b374a36f4892-kube-api-access-6drfg" (OuterVolumeSpecName: "kube-api-access-6drfg") pod "667ef294-7065-410a-81fe-b374a36f4892" (UID: "667ef294-7065-410a-81fe-b374a36f4892"). InnerVolumeSpecName "kube-api-access-6drfg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:53:29.002526 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:29.002506 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz5hs\" (UniqueName: \"kubernetes.io/projected/2e83c0a9-9f0b-4268-8369-d49bc08b8d32-kube-api-access-bz5hs\") pod \"authorino-9bc8d787b-pg5hw\" (UID: \"2e83c0a9-9f0b-4268-8369-d49bc08b8d32\") " pod="kuadrant-system/authorino-9bc8d787b-pg5hw" Jul 10 14:53:29.093881 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:29.093838 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6drfg\" (UniqueName: \"kubernetes.io/projected/667ef294-7065-410a-81fe-b374a36f4892-kube-api-access-6drfg\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:53:29.841690 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:29.841655 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bc8d787b-pg5hw" Jul 10 14:53:29.842088 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:29.841655 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vx5fh" Jul 10 14:53:29.849672 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:29.849650 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bc8d787b-pg5hw" Jul 10 14:53:29.890009 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:29.889978 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vx5fh"] Jul 10 14:53:29.894963 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:29.894936 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vx5fh"] Jul 10 14:53:30.002902 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:30.002861 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz5hs\" (UniqueName: \"kubernetes.io/projected/2e83c0a9-9f0b-4268-8369-d49bc08b8d32-kube-api-access-bz5hs\") pod \"2e83c0a9-9f0b-4268-8369-d49bc08b8d32\" (UID: \"2e83c0a9-9f0b-4268-8369-d49bc08b8d32\") " Jul 10 14:53:30.005103 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:30.005076 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e83c0a9-9f0b-4268-8369-d49bc08b8d32-kube-api-access-bz5hs" (OuterVolumeSpecName: "kube-api-access-bz5hs") pod "2e83c0a9-9f0b-4268-8369-d49bc08b8d32" (UID: "2e83c0a9-9f0b-4268-8369-d49bc08b8d32"). InnerVolumeSpecName "kube-api-access-bz5hs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:53:30.104293 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:30.104209 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz5hs\" (UniqueName: \"kubernetes.io/projected/2e83c0a9-9f0b-4268-8369-d49bc08b8d32-kube-api-access-bz5hs\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:53:30.279500 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:30.279469 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="667ef294-7065-410a-81fe-b374a36f4892" path="/var/lib/kubelet/pods/667ef294-7065-410a-81fe-b374a36f4892/volumes" Jul 10 14:53:30.847355 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:30.847309 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bc8d787b-pg5hw" Jul 10 14:53:30.885097 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:30.885066 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9bc8d787b-pg5hw"] Jul 10 14:53:30.894311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:30.894282 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-9bc8d787b-pg5hw"] Jul 10 14:53:32.279515 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:32.279486 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e83c0a9-9f0b-4268-8369-d49bc08b8d32" path="/var/lib/kubelet/pods/2e83c0a9-9f0b-4268-8369-d49bc08b8d32/volumes" Jul 10 14:53:33.059283 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.059244 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zjlzh"] Jul 10 14:53:33.059507 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.059484 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-zjlzh" podUID="6534cd6b-26dc-4537-8417-63ed7a7ac19b" containerName="authorino" containerID="cri-o://b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2" gracePeriod=30 Jul 10 14:53:33.359700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.359670 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zjlzh" Jul 10 14:53:33.535618 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.535579 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q646k\" (UniqueName: \"kubernetes.io/projected/6534cd6b-26dc-4537-8417-63ed7a7ac19b-kube-api-access-q646k\") pod \"6534cd6b-26dc-4537-8417-63ed7a7ac19b\" (UID: \"6534cd6b-26dc-4537-8417-63ed7a7ac19b\") " Jul 10 14:53:33.537771 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.537739 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6534cd6b-26dc-4537-8417-63ed7a7ac19b-kube-api-access-q646k" (OuterVolumeSpecName: "kube-api-access-q646k") pod "6534cd6b-26dc-4537-8417-63ed7a7ac19b" (UID: "6534cd6b-26dc-4537-8417-63ed7a7ac19b"). InnerVolumeSpecName "kube-api-access-q646k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:53:33.636815 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.636744 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q646k\" (UniqueName: \"kubernetes.io/projected/6534cd6b-26dc-4537-8417-63ed7a7ac19b-kube-api-access-q646k\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:53:33.866283 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.866245 2562 generic.go:356] "Generic (PLEG): container finished" podID="6534cd6b-26dc-4537-8417-63ed7a7ac19b" containerID="b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2" exitCode=0 Jul 10 14:53:33.866497 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.866290 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zjlzh" Jul 10 14:53:33.866497 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.866350 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zjlzh" event={"ID":"6534cd6b-26dc-4537-8417-63ed7a7ac19b","Type":"ContainerDied","Data":"b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2"} Jul 10 14:53:33.866497 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.866393 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zjlzh" event={"ID":"6534cd6b-26dc-4537-8417-63ed7a7ac19b","Type":"ContainerDied","Data":"b3771309e54ae584c25322dd4b0f36e7651f00866aaae7078a66fef78bfab0db"} Jul 10 14:53:33.866497 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.866415 2562 scope.go:117] "RemoveContainer" containerID="b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2" Jul 10 14:53:33.876288 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.876266 2562 scope.go:117] "RemoveContainer" containerID="b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2" Jul 10 14:53:33.876590 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:53:33.876573 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2\": container with ID starting with b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2 not found: ID does not exist" containerID="b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2" Jul 10 14:53:33.876670 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.876598 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2"} err="failed to get container status \"b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2\": rpc error: code = NotFound desc = could not find container \"b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2\": container with ID starting with b0004020c4ac0cfcdb120095a255a67518a6458b9feaa63849c2fa980d1d88e2 not found: ID does not exist" Jul 10 14:53:33.900890 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.900808 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zjlzh"] Jul 10 14:53:33.910625 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:33.910593 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zjlzh"] Jul 10 14:53:34.280092 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:34.280055 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6534cd6b-26dc-4537-8417-63ed7a7ac19b" path="/var/lib/kubelet/pods/6534cd6b-26dc-4537-8417-63ed7a7ac19b/volumes" Jul 10 14:53:38.047835 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.047800 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-585b678f49-2k4qp"] Jul 10 14:53:38.048303 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.048287 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6534cd6b-26dc-4537-8417-63ed7a7ac19b" containerName="authorino" Jul 10 14:53:38.048389 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.048305 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6534cd6b-26dc-4537-8417-63ed7a7ac19b" containerName="authorino" Jul 10 14:53:38.048427 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.048409 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6534cd6b-26dc-4537-8417-63ed7a7ac19b" containerName="authorino" Jul 10 14:53:38.058589 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.058560 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:38.065773 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.065741 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 14:53:38.065990 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.065838 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-scqc9\"" Jul 10 14:53:38.066700 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.066677 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 14:53:38.069680 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.069653 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-585b678f49-2k4qp"] Jul 10 14:53:38.174803 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.174757 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28fbb109-0ab9-46df-9078-62a9223e94ee-webhook-cert\") pod \"maas-controller-585b678f49-2k4qp\" (UID: \"28fbb109-0ab9-46df-9078-62a9223e94ee\") " pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:38.174803 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.174799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpsx8\" (UniqueName: \"kubernetes.io/projected/28fbb109-0ab9-46df-9078-62a9223e94ee-kube-api-access-hpsx8\") pod \"maas-controller-585b678f49-2k4qp\" (UID: \"28fbb109-0ab9-46df-9078-62a9223e94ee\") " pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:38.258679 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.258646 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68f867f7b6-b7b47"] Jul 10 14:53:38.261015 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.260999 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:53:38.275862 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.275833 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28fbb109-0ab9-46df-9078-62a9223e94ee-webhook-cert\") pod \"maas-controller-585b678f49-2k4qp\" (UID: \"28fbb109-0ab9-46df-9078-62a9223e94ee\") " pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:38.276354 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.276310 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpsx8\" (UniqueName: \"kubernetes.io/projected/28fbb109-0ab9-46df-9078-62a9223e94ee-kube-api-access-hpsx8\") pod \"maas-controller-585b678f49-2k4qp\" (UID: \"28fbb109-0ab9-46df-9078-62a9223e94ee\") " pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:38.279259 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.279231 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28fbb109-0ab9-46df-9078-62a9223e94ee-webhook-cert\") pod \"maas-controller-585b678f49-2k4qp\" (UID: \"28fbb109-0ab9-46df-9078-62a9223e94ee\") " pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:38.281447 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.281424 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68f867f7b6-b7b47"] Jul 10 14:53:38.303256 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.303185 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpsx8\" (UniqueName: \"kubernetes.io/projected/28fbb109-0ab9-46df-9078-62a9223e94ee-kube-api-access-hpsx8\") pod \"maas-controller-585b678f49-2k4qp\" (UID: \"28fbb109-0ab9-46df-9078-62a9223e94ee\") " pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:38.377405 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.377367 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:38.377625 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.377596 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a21dacd-b810-45f0-a59f-5ab15645b575-webhook-cert\") pod \"maas-controller-68f867f7b6-b7b47\" (UID: \"6a21dacd-b810-45f0-a59f-5ab15645b575\") " pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:53:38.377686 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.377642 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r26vl\" (UniqueName: \"kubernetes.io/projected/6a21dacd-b810-45f0-a59f-5ab15645b575-kube-api-access-r26vl\") pod \"maas-controller-68f867f7b6-b7b47\" (UID: \"6a21dacd-b810-45f0-a59f-5ab15645b575\") " pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:53:38.479137 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.479086 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a21dacd-b810-45f0-a59f-5ab15645b575-webhook-cert\") pod \"maas-controller-68f867f7b6-b7b47\" (UID: \"6a21dacd-b810-45f0-a59f-5ab15645b575\") " pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:53:38.479359 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.479151 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r26vl\" (UniqueName: \"kubernetes.io/projected/6a21dacd-b810-45f0-a59f-5ab15645b575-kube-api-access-r26vl\") pod \"maas-controller-68f867f7b6-b7b47\" (UID: \"6a21dacd-b810-45f0-a59f-5ab15645b575\") " pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:53:38.481987 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.481956 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a21dacd-b810-45f0-a59f-5ab15645b575-webhook-cert\") pod \"maas-controller-68f867f7b6-b7b47\" (UID: \"6a21dacd-b810-45f0-a59f-5ab15645b575\") " pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:53:38.492916 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.492885 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r26vl\" (UniqueName: \"kubernetes.io/projected/6a21dacd-b810-45f0-a59f-5ab15645b575-kube-api-access-r26vl\") pod \"maas-controller-68f867f7b6-b7b47\" (UID: \"6a21dacd-b810-45f0-a59f-5ab15645b575\") " pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:53:38.522886 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.522862 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-585b678f49-2k4qp"] Jul 10 14:53:38.524870 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:53:38.524847 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28fbb109_0ab9_46df_9078_62a9223e94ee.slice/crio-33d8db72bbb9b965c066d7f43dd3391c11c239ad676d1cf2295cbd87128c902c WatchSource:0}: Error finding container 33d8db72bbb9b965c066d7f43dd3391c11c239ad676d1cf2295cbd87128c902c: Status 404 returned error can't find the container with id 33d8db72bbb9b965c066d7f43dd3391c11c239ad676d1cf2295cbd87128c902c Jul 10 14:53:38.571820 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.571720 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:53:38.713117 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.713088 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68f867f7b6-b7b47"] Jul 10 14:53:38.715264 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:53:38.715235 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a21dacd_b810_45f0_a59f_5ab15645b575.slice/crio-a1caa1744466be80754f6bd686c47febe7c1fbb2781e6a8634e8b45135ed033c WatchSource:0}: Error finding container a1caa1744466be80754f6bd686c47febe7c1fbb2781e6a8634e8b45135ed033c: Status 404 returned error can't find the container with id a1caa1744466be80754f6bd686c47febe7c1fbb2781e6a8634e8b45135ed033c Jul 10 14:53:38.895538 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.895434 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68f867f7b6-b7b47" event={"ID":"6a21dacd-b810-45f0-a59f-5ab15645b575","Type":"ContainerStarted","Data":"a1caa1744466be80754f6bd686c47febe7c1fbb2781e6a8634e8b45135ed033c"} Jul 10 14:53:38.896463 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:38.896432 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585b678f49-2k4qp" event={"ID":"28fbb109-0ab9-46df-9078-62a9223e94ee","Type":"ContainerStarted","Data":"33d8db72bbb9b965c066d7f43dd3391c11c239ad676d1cf2295cbd87128c902c"} Jul 10 14:53:41.920409 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:41.920368 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68f867f7b6-b7b47" event={"ID":"6a21dacd-b810-45f0-a59f-5ab15645b575","Type":"ContainerStarted","Data":"73f39cf182d9bbcea270e98ca293ae2b6106296f224d012eee87cec1148f04fe"} Jul 10 14:53:41.920860 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:41.920444 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:53:41.921791 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:41.921755 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585b678f49-2k4qp" event={"ID":"28fbb109-0ab9-46df-9078-62a9223e94ee","Type":"ContainerStarted","Data":"90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4"} Jul 10 14:53:41.921898 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:41.921815 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:41.938949 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:41.938904 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68f867f7b6-b7b47" podStartSLOduration=1.6651945590000001 podStartE2EDuration="3.938890713s" totalImagesPullingTime="2.273696154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:53:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:53:41.936646685 +0000 UTC m=+502.217161383" watchObservedRunningTime="2026-07-10 14:53:41.938890713 +0000 UTC m=+502.219405410" Jul 10 14:53:41.959379 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:41.959327 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-585b678f49-2k4qp" podStartSLOduration=1.490099475 podStartE2EDuration="3.959314727s" totalImagesPullingTime="2.469215252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:53:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:53:41.956572435 +0000 UTC m=+502.237087157" watchObservedRunningTime="2026-07-10 14:53:41.959314727 +0000 UTC m=+502.239829425" Jul 10 14:53:52.934353 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:52.934302 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:53:52.934729 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:52.934502 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:53.002014 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.001977 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-585b678f49-2k4qp"] Jul 10 14:53:53.002321 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.002259 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-585b678f49-2k4qp" podUID="28fbb109-0ab9-46df-9078-62a9223e94ee" containerName="manager" containerID="cri-o://90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4" gracePeriod=30 Jul 10 14:53:53.246413 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.246384 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:53.309636 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.309602 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpsx8\" (UniqueName: \"kubernetes.io/projected/28fbb109-0ab9-46df-9078-62a9223e94ee-kube-api-access-hpsx8\") pod \"28fbb109-0ab9-46df-9078-62a9223e94ee\" (UID: \"28fbb109-0ab9-46df-9078-62a9223e94ee\") " Jul 10 14:53:53.309845 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.309654 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28fbb109-0ab9-46df-9078-62a9223e94ee-webhook-cert\") pod \"28fbb109-0ab9-46df-9078-62a9223e94ee\" (UID: \"28fbb109-0ab9-46df-9078-62a9223e94ee\") " Jul 10 14:53:53.317123 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.313433 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28fbb109-0ab9-46df-9078-62a9223e94ee-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "28fbb109-0ab9-46df-9078-62a9223e94ee" (UID: "28fbb109-0ab9-46df-9078-62a9223e94ee"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:53:53.317123 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.315487 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28fbb109-0ab9-46df-9078-62a9223e94ee-kube-api-access-hpsx8" (OuterVolumeSpecName: "kube-api-access-hpsx8") pod "28fbb109-0ab9-46df-9078-62a9223e94ee" (UID: "28fbb109-0ab9-46df-9078-62a9223e94ee"). InnerVolumeSpecName "kube-api-access-hpsx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:53:53.411041 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.411007 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpsx8\" (UniqueName: \"kubernetes.io/projected/28fbb109-0ab9-46df-9078-62a9223e94ee-kube-api-access-hpsx8\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:53:53.411041 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.411036 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28fbb109-0ab9-46df-9078-62a9223e94ee-webhook-cert\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:53:53.993803 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.993766 2562 generic.go:356] "Generic (PLEG): container finished" podID="28fbb109-0ab9-46df-9078-62a9223e94ee" containerID="90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4" exitCode=0 Jul 10 14:53:53.994220 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.993832 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585b678f49-2k4qp" Jul 10 14:53:53.994220 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.993859 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585b678f49-2k4qp" event={"ID":"28fbb109-0ab9-46df-9078-62a9223e94ee","Type":"ContainerDied","Data":"90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4"} Jul 10 14:53:53.994220 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.993937 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585b678f49-2k4qp" event={"ID":"28fbb109-0ab9-46df-9078-62a9223e94ee","Type":"ContainerDied","Data":"33d8db72bbb9b965c066d7f43dd3391c11c239ad676d1cf2295cbd87128c902c"} Jul 10 14:53:53.994220 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:53.993960 2562 scope.go:117] "RemoveContainer" containerID="90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4" Jul 10 14:53:54.006012 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:54.005847 2562 scope.go:117] "RemoveContainer" containerID="90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4" Jul 10 14:53:54.006380 ip-10-0-135-1 kubenswrapper[2562]: E0710 14:53:54.006358 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4\": container with ID starting with 90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4 not found: ID does not exist" containerID="90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4" Jul 10 14:53:54.006467 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:54.006389 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4"} err="failed to get container status \"90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4\": rpc error: code = NotFound desc = could not find container \"90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4\": container with ID starting with 90380f1586fcb8dc10e450bed2a72f7a0829fbf1fb00895b435a9fbc2a806ef4 not found: ID does not exist" Jul 10 14:53:54.017004 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:54.016979 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-585b678f49-2k4qp"] Jul 10 14:53:54.020941 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:54.020918 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-585b678f49-2k4qp"] Jul 10 14:53:54.280299 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:53:54.280270 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28fbb109-0ab9-46df-9078-62a9223e94ee" path="/var/lib/kubelet/pods/28fbb109-0ab9-46df-9078-62a9223e94ee/volumes" Jul 10 14:54:14.921910 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:14.921839 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4"] Jul 10 14:54:14.922407 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:14.922320 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28fbb109-0ab9-46df-9078-62a9223e94ee" containerName="manager" Jul 10 14:54:14.922407 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:14.922367 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="28fbb109-0ab9-46df-9078-62a9223e94ee" containerName="manager" Jul 10 14:54:14.922486 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:14.922455 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="28fbb109-0ab9-46df-9078-62a9223e94ee" containerName="manager" Jul 10 14:54:14.926232 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:14.926209 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:14.929640 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:14.929620 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-whm6h\"" Jul 10 14:54:14.935409 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:14.935391 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 10 14:54:14.950893 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:14.950866 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4"] Jul 10 14:54:15.006545 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.006508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgkfp\" (UniqueName: \"kubernetes.io/projected/4039ac58-0e7f-4c22-814c-bf7afa939570-kube-api-access-hgkfp\") pod \"maas-api-6b5665c595-g9cf4\" (UID: \"4039ac58-0e7f-4c22-814c-bf7afa939570\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:15.006729 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.006598 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4039ac58-0e7f-4c22-814c-bf7afa939570-maas-api-tls\") pod \"maas-api-6b5665c595-g9cf4\" (UID: \"4039ac58-0e7f-4c22-814c-bf7afa939570\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:15.006729 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.006702 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4039ac58-0e7f-4c22-814c-bf7afa939570-openshift-service-ca\") pod \"maas-api-6b5665c595-g9cf4\" (UID: \"4039ac58-0e7f-4c22-814c-bf7afa939570\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:15.108202 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.108169 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4039ac58-0e7f-4c22-814c-bf7afa939570-maas-api-tls\") pod \"maas-api-6b5665c595-g9cf4\" (UID: \"4039ac58-0e7f-4c22-814c-bf7afa939570\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:15.108400 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.108235 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4039ac58-0e7f-4c22-814c-bf7afa939570-openshift-service-ca\") pod \"maas-api-6b5665c595-g9cf4\" (UID: \"4039ac58-0e7f-4c22-814c-bf7afa939570\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:15.108400 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.108256 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgkfp\" (UniqueName: \"kubernetes.io/projected/4039ac58-0e7f-4c22-814c-bf7afa939570-kube-api-access-hgkfp\") pod \"maas-api-6b5665c595-g9cf4\" (UID: \"4039ac58-0e7f-4c22-814c-bf7afa939570\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:15.108991 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.108971 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4039ac58-0e7f-4c22-814c-bf7afa939570-openshift-service-ca\") pod \"maas-api-6b5665c595-g9cf4\" (UID: \"4039ac58-0e7f-4c22-814c-bf7afa939570\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:15.110750 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.110727 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4039ac58-0e7f-4c22-814c-bf7afa939570-maas-api-tls\") pod \"maas-api-6b5665c595-g9cf4\" (UID: \"4039ac58-0e7f-4c22-814c-bf7afa939570\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:15.127915 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.127890 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgkfp\" (UniqueName: \"kubernetes.io/projected/4039ac58-0e7f-4c22-814c-bf7afa939570-kube-api-access-hgkfp\") pod \"maas-api-6b5665c595-g9cf4\" (UID: \"4039ac58-0e7f-4c22-814c-bf7afa939570\") " pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:15.237138 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.237044 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:15.386061 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.386037 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4"] Jul 10 14:54:15.388347 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:54:15.388300 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4039ac58_0e7f_4c22_814c_bf7afa939570.slice/crio-9e2bb734c8abd4032e14776413d1f022ac240b80a7d3ba235af046a9cd502519 WatchSource:0}: Error finding container 9e2bb734c8abd4032e14776413d1f022ac240b80a7d3ba235af046a9cd502519: Status 404 returned error can't find the container with id 9e2bb734c8abd4032e14776413d1f022ac240b80a7d3ba235af046a9cd502519 Jul 10 14:54:15.707836 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.707804 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-z4pgz"] Jul 10 14:54:15.713143 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.713127 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" Jul 10 14:54:15.716883 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.716863 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-82txg\"" Jul 10 14:54:15.719994 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.719974 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 14:54:15.729139 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.729107 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:54:15.731571 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.731543 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-z4pgz"] Jul 10 14:54:15.781764 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.781736 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6hs5q"] Jul 10 14:54:15.785680 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.785665 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" Jul 10 14:54:15.798573 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.798546 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6hs5q"] Jul 10 14:54:15.814870 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.814832 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndsw5\" (UniqueName: \"kubernetes.io/projected/b758d2e9-5dfd-4889-8700-13e1703ed785-kube-api-access-ndsw5\") pod \"payload-pre-processing-7964457696-z4pgz\" (UID: \"b758d2e9-5dfd-4889-8700-13e1703ed785\") " pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" Jul 10 14:54:15.815044 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.814892 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b758d2e9-5dfd-4889-8700-13e1703ed785-plugins-config-volume\") pod \"payload-pre-processing-7964457696-z4pgz\" (UID: \"b758d2e9-5dfd-4889-8700-13e1703ed785\") " pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" Jul 10 14:54:15.916040 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.916002 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndsw5\" (UniqueName: \"kubernetes.io/projected/b758d2e9-5dfd-4889-8700-13e1703ed785-kube-api-access-ndsw5\") pod \"payload-pre-processing-7964457696-z4pgz\" (UID: \"b758d2e9-5dfd-4889-8700-13e1703ed785\") " pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" Jul 10 14:54:15.916040 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.916048 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/356d1473-1d2c-4ab0-87cc-5b73e768848e-plugins-config-volume\") pod \"payload-processing-58f86c7798-6hs5q\" (UID: \"356d1473-1d2c-4ab0-87cc-5b73e768848e\") " pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" Jul 10 14:54:15.916262 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.916072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wlv9\" (UniqueName: \"kubernetes.io/projected/356d1473-1d2c-4ab0-87cc-5b73e768848e-kube-api-access-2wlv9\") pod \"payload-processing-58f86c7798-6hs5q\" (UID: \"356d1473-1d2c-4ab0-87cc-5b73e768848e\") " pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" Jul 10 14:54:15.916262 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.916099 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b758d2e9-5dfd-4889-8700-13e1703ed785-plugins-config-volume\") pod \"payload-pre-processing-7964457696-z4pgz\" (UID: \"b758d2e9-5dfd-4889-8700-13e1703ed785\") " pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" Jul 10 14:54:15.916803 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.916782 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b758d2e9-5dfd-4889-8700-13e1703ed785-plugins-config-volume\") pod \"payload-pre-processing-7964457696-z4pgz\" (UID: \"b758d2e9-5dfd-4889-8700-13e1703ed785\") " pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" Jul 10 14:54:15.926247 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:15.926221 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndsw5\" (UniqueName: \"kubernetes.io/projected/b758d2e9-5dfd-4889-8700-13e1703ed785-kube-api-access-ndsw5\") pod \"payload-pre-processing-7964457696-z4pgz\" (UID: \"b758d2e9-5dfd-4889-8700-13e1703ed785\") " pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" Jul 10 14:54:16.017045 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:16.017009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/356d1473-1d2c-4ab0-87cc-5b73e768848e-plugins-config-volume\") pod \"payload-processing-58f86c7798-6hs5q\" (UID: \"356d1473-1d2c-4ab0-87cc-5b73e768848e\") " pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" Jul 10 14:54:16.017045 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:16.017044 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wlv9\" (UniqueName: \"kubernetes.io/projected/356d1473-1d2c-4ab0-87cc-5b73e768848e-kube-api-access-2wlv9\") pod \"payload-processing-58f86c7798-6hs5q\" (UID: \"356d1473-1d2c-4ab0-87cc-5b73e768848e\") " pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" Jul 10 14:54:16.017779 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:16.017752 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/356d1473-1d2c-4ab0-87cc-5b73e768848e-plugins-config-volume\") pod \"payload-processing-58f86c7798-6hs5q\" (UID: \"356d1473-1d2c-4ab0-87cc-5b73e768848e\") " pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" Jul 10 14:54:16.023868 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:16.023851 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" Jul 10 14:54:16.030512 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:16.030481 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wlv9\" (UniqueName: \"kubernetes.io/projected/356d1473-1d2c-4ab0-87cc-5b73e768848e-kube-api-access-2wlv9\") pod \"payload-processing-58f86c7798-6hs5q\" (UID: \"356d1473-1d2c-4ab0-87cc-5b73e768848e\") " pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" Jul 10 14:54:16.095970 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:16.095937 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" Jul 10 14:54:16.118491 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:16.118310 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" event={"ID":"4039ac58-0e7f-4c22-814c-bf7afa939570","Type":"ContainerStarted","Data":"9e2bb734c8abd4032e14776413d1f022ac240b80a7d3ba235af046a9cd502519"} Jul 10 14:54:16.169941 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:16.169915 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-z4pgz"] Jul 10 14:54:16.170948 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:54:16.170918 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb758d2e9_5dfd_4889_8700_13e1703ed785.slice/crio-9322a5dd43681971bbfdb6cb74d3af54fa8d636c70629ec7eb623bd57747daa9 WatchSource:0}: Error finding container 9322a5dd43681971bbfdb6cb74d3af54fa8d636c70629ec7eb623bd57747daa9: Status 404 returned error can't find the container with id 9322a5dd43681971bbfdb6cb74d3af54fa8d636c70629ec7eb623bd57747daa9 Jul 10 14:54:16.250691 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:16.250663 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6hs5q"] Jul 10 14:54:16.251734 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:54:16.251709 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod356d1473_1d2c_4ab0_87cc_5b73e768848e.slice/crio-dfbce8179422c3fde6b9e1278febd62f164d063810028cb724498860b2e427b3 WatchSource:0}: Error finding container dfbce8179422c3fde6b9e1278febd62f164d063810028cb724498860b2e427b3: Status 404 returned error can't find the container with id dfbce8179422c3fde6b9e1278febd62f164d063810028cb724498860b2e427b3 Jul 10 14:54:17.129373 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:17.129318 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" event={"ID":"356d1473-1d2c-4ab0-87cc-5b73e768848e","Type":"ContainerStarted","Data":"dfbce8179422c3fde6b9e1278febd62f164d063810028cb724498860b2e427b3"} Jul 10 14:54:17.131901 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:17.131833 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" event={"ID":"b758d2e9-5dfd-4889-8700-13e1703ed785","Type":"ContainerStarted","Data":"9322a5dd43681971bbfdb6cb74d3af54fa8d636c70629ec7eb623bd57747daa9"} Jul 10 14:54:18.139300 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:18.139203 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" event={"ID":"356d1473-1d2c-4ab0-87cc-5b73e768848e","Type":"ContainerStarted","Data":"413e851dc8ab3b494a1fbd2edf6252de5176afc9dd35523cab58158a9ab05f74"} Jul 10 14:54:18.139740 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:18.139327 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" Jul 10 14:54:18.140920 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:18.140888 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" event={"ID":"b758d2e9-5dfd-4889-8700-13e1703ed785","Type":"ContainerStarted","Data":"78977a5827ad80e7a0d7c8a01484fceeb80738d6d7c277c2f1665ef63b857136"} Jul 10 14:54:18.141077 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:18.140937 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" Jul 10 14:54:18.142306 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:18.142286 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" event={"ID":"4039ac58-0e7f-4c22-814c-bf7afa939570","Type":"ContainerStarted","Data":"cb2cd290828b2b9e01f61810477a952414b358097c3ac4f8cb7b19bd47f16f92"} Jul 10 14:54:18.142460 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:18.142402 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:18.174020 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:18.173941 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" podStartSLOduration=1.5418978669999999 podStartE2EDuration="3.173919019s" totalImagesPullingTime="1.632021152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:54:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:54:18.1703925 +0000 UTC m=+538.450907193" watchObservedRunningTime="2026-07-10 14:54:18.173919019 +0000 UTC m=+538.454433719" Jul 10 14:54:18.191496 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:18.191434 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" podStartSLOduration=1.698026974 podStartE2EDuration="4.19141175s" totalImagesPullingTime="2.493384776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:54:18.190940886 +0000 UTC m=+538.471455585" watchObservedRunningTime="2026-07-10 14:54:18.19141175 +0000 UTC m=+538.471926450" Jul 10 14:54:18.208438 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:18.208388 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" podStartSLOduration=1.498241348 podStartE2EDuration="3.208370251s" totalImagesPullingTime="1.710128903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:54:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:54:18.206300131 +0000 UTC m=+538.486814828" watchObservedRunningTime="2026-07-10 14:54:18.208370251 +0000 UTC m=+538.488884951" Jul 10 14:54:24.153827 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:24.153797 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b5665c595-g9cf4" Jul 10 14:54:29.150874 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:29.150841 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-6hs5q" Jul 10 14:54:29.151290 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:29.150925 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-z4pgz" Jul 10 14:54:30.172477 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:30.172445 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:54:42.229360 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.229310 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd"] Jul 10 14:54:42.234218 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.234193 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.236733 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.236708 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 14:54:42.236862 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.236756 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 14:54:42.237551 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.237522 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 10 14:54:42.237551 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.237543 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wk7h7\"" Jul 10 14:54:42.243656 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.243632 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd"] Jul 10 14:54:42.353393 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.353357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmh4r\" (UniqueName: \"kubernetes.io/projected/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-kube-api-access-wmh4r\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.353393 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.353392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.353613 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.353409 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.353613 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.353492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.353613 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.353534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.353613 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.353562 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.454723 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.454689 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.454890 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.454745 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmh4r\" (UniqueName: \"kubernetes.io/projected/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-kube-api-access-wmh4r\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.454890 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.454768 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.454890 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.454783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.454890 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.454824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.454890 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.454861 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.455223 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.455197 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.455291 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.455252 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.455381 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.455280 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.457160 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.457140 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.457263 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.457212 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.465186 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.465159 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmh4r\" (UniqueName: \"kubernetes.io/projected/2bfa1839-0882-4cb7-b409-dfdcc9e369f5-kube-api-access-wmh4r\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zfnmd\" (UID: \"2bfa1839-0882-4cb7-b409-dfdcc9e369f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.546232 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.546194 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:42.684058 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:42.684036 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd"] Jul 10 14:54:42.686306 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:54:42.686277 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bfa1839_0882_4cb7_b409_dfdcc9e369f5.slice/crio-d647ecf05220f8583fba1af918d9ddbbc00d0c4059337137d2d2f37b8cc8ea1f WatchSource:0}: Error finding container d647ecf05220f8583fba1af918d9ddbbc00d0c4059337137d2d2f37b8cc8ea1f: Status 404 returned error can't find the container with id d647ecf05220f8583fba1af918d9ddbbc00d0c4059337137d2d2f37b8cc8ea1f Jul 10 14:54:43.015320 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:43.015164 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:54:43.294774 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:43.294669 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" event={"ID":"2bfa1839-0882-4cb7-b409-dfdcc9e369f5","Type":"ContainerStarted","Data":"d647ecf05220f8583fba1af918d9ddbbc00d0c4059337137d2d2f37b8cc8ea1f"} Jul 10 14:54:44.301641 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:44.301609 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" event={"ID":"2bfa1839-0882-4cb7-b409-dfdcc9e369f5","Type":"ContainerStarted","Data":"8fa8a16a3e74bcdf39f8f06c2230467a19ccc166101493ed90d24c6fd088e9b6"} Jul 10 14:54:44.330217 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:44.330144 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" podStartSLOduration=1.267743887 podStartE2EDuration="2.330122541s" totalImagesPullingTime="1.062378654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:54:44.325358402 +0000 UTC m=+564.605873102" watchObservedRunningTime="2026-07-10 14:54:44.330122541 +0000 UTC m=+564.610637243" Jul 10 14:54:52.546534 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:52.546490 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:52.547053 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:52.546566 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:52.559095 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:52.559067 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:53.371521 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:53.371494 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zfnmd" Jul 10 14:54:53.808924 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:54:53.808889 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:55:02.498820 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.498783 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87"] Jul 10 14:55:02.504531 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.504510 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.507980 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.507958 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 10 14:55:02.524650 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.524625 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87"] Jul 10 14:55:02.643005 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.642964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm4vq\" (UniqueName: \"kubernetes.io/projected/95ccaef0-b406-4139-891b-afe9d5cc1c88-kube-api-access-pm4vq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.643190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.643021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.643190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.643042 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.643190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.643137 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.643190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.643170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95ccaef0-b406-4139-891b-afe9d5cc1c88-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.643190 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.643189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.744112 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.744069 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.744112 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.744117 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95ccaef0-b406-4139-891b-afe9d5cc1c88-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.744413 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.744144 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.744413 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.744205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm4vq\" (UniqueName: \"kubernetes.io/projected/95ccaef0-b406-4139-891b-afe9d5cc1c88-kube-api-access-pm4vq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.744413 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.744261 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.744413 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.744283 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.744683 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.744658 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.744751 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.744695 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.744751 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.744720 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.746678 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.746653 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95ccaef0-b406-4139-891b-afe9d5cc1c88-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.746905 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.746887 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95ccaef0-b406-4139-891b-afe9d5cc1c88-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.753222 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.753163 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm4vq\" (UniqueName: \"kubernetes.io/projected/95ccaef0-b406-4139-891b-afe9d5cc1c88-kube-api-access-pm4vq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87\" (UID: \"95ccaef0-b406-4139-891b-afe9d5cc1c88\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.814823 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.814784 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:02.962032 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:02.961990 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87"] Jul 10 14:55:02.968617 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:55:02.968579 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95ccaef0_b406_4139_891b_afe9d5cc1c88.slice/crio-7730eaa5f2c5f77fa35b746f6cda6699878d52eed9cf1b1676e20ead7069f7d5 WatchSource:0}: Error finding container 7730eaa5f2c5f77fa35b746f6cda6699878d52eed9cf1b1676e20ead7069f7d5: Status 404 returned error can't find the container with id 7730eaa5f2c5f77fa35b746f6cda6699878d52eed9cf1b1676e20ead7069f7d5 Jul 10 14:55:03.411216 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:03.411184 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" event={"ID":"95ccaef0-b406-4139-891b-afe9d5cc1c88","Type":"ContainerStarted","Data":"78e188a2631f9b6635827a12359a47d8f9a53c31ee2b9489c544e8baade708e4"} Jul 10 14:55:03.411443 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:03.411239 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" event={"ID":"95ccaef0-b406-4139-891b-afe9d5cc1c88","Type":"ContainerStarted","Data":"7730eaa5f2c5f77fa35b746f6cda6699878d52eed9cf1b1676e20ead7069f7d5"} Jul 10 14:55:03.434821 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:03.434764 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" podStartSLOduration=1.086693737 podStartE2EDuration="1.434746429s" totalImagesPullingTime="348.052692ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:55:03.431093004 +0000 UTC m=+583.711607701" watchObservedRunningTime="2026-07-10 14:55:03.434746429 +0000 UTC m=+583.715261127" Jul 10 14:55:04.016379 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:04.016316 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:55:12.815807 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:12.815767 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:12.816270 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:12.815831 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:12.828533 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:12.828509 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:13.482524 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:13.482496 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87" Jul 10 14:55:15.208922 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.208888 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2"] Jul 10 14:55:15.215371 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.215314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.217990 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.217962 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 10 14:55:15.225579 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.225556 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2"] Jul 10 14:55:15.256284 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.256254 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.256443 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.256353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67aee487-6e3a-41a9-9cb9-3eae80024733-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.256443 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.256379 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.256443 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.256400 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.256566 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.256472 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.256566 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.256512 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g67gn\" (UniqueName: \"kubernetes.io/projected/67aee487-6e3a-41a9-9cb9-3eae80024733-kube-api-access-g67gn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.357674 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.357633 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.357870 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.357685 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g67gn\" (UniqueName: \"kubernetes.io/projected/67aee487-6e3a-41a9-9cb9-3eae80024733-kube-api-access-g67gn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.357870 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.357729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.357870 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.357781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67aee487-6e3a-41a9-9cb9-3eae80024733-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.357870 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.357813 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.357870 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.357838 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.358158 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.358137 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.358222 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.358136 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.358222 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.358136 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.360388 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.360360 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67aee487-6e3a-41a9-9cb9-3eae80024733-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.360592 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.360578 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67aee487-6e3a-41a9-9cb9-3eae80024733-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.367414 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.367392 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g67gn\" (UniqueName: \"kubernetes.io/projected/67aee487-6e3a-41a9-9cb9-3eae80024733-kube-api-access-g67gn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6skm2\" (UID: \"67aee487-6e3a-41a9-9cb9-3eae80024733\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.527422 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.527383 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:15.670641 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:15.670617 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2"] Jul 10 14:55:15.672304 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:55:15.672273 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67aee487_6e3a_41a9_9cb9_3eae80024733.slice/crio-8bba0e5680cf037012a80f44cf824e8a8f928b6d2a8a56b3220a69aa7654d2ad WatchSource:0}: Error finding container 8bba0e5680cf037012a80f44cf824e8a8f928b6d2a8a56b3220a69aa7654d2ad: Status 404 returned error can't find the container with id 8bba0e5680cf037012a80f44cf824e8a8f928b6d2a8a56b3220a69aa7654d2ad Jul 10 14:55:16.488627 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:16.488542 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" event={"ID":"67aee487-6e3a-41a9-9cb9-3eae80024733","Type":"ContainerStarted","Data":"ed02facd2e9faea940071e60566df493d7d92295afe60c3b085d10282f57e971"} Jul 10 14:55:16.488627 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:16.488592 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" event={"ID":"67aee487-6e3a-41a9-9cb9-3eae80024733","Type":"ContainerStarted","Data":"8bba0e5680cf037012a80f44cf824e8a8f928b6d2a8a56b3220a69aa7654d2ad"} Jul 10 14:55:16.517488 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:16.517442 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" podStartSLOduration=1.050821854 podStartE2EDuration="1.517426692s" totalImagesPullingTime="466.604838ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:55:16.513893295 +0000 UTC m=+596.794407993" watchObservedRunningTime="2026-07-10 14:55:16.517426692 +0000 UTC m=+596.797941390" Jul 10 14:55:17.524549 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:17.521230 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:55:18.034866 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.034836 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9"] Jul 10 14:55:18.038799 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.038784 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.042230 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.042206 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 10 14:55:18.059605 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.059574 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9"] Jul 10 14:55:18.082220 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.082185 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.082405 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.082299 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb5rw\" (UniqueName: \"kubernetes.io/projected/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-kube-api-access-vb5rw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.082405 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.082358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.082539 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.082418 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.082539 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.082501 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.082539 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.082532 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.183203 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.183166 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.183203 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.183207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.183487 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.183239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.183487 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.183284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb5rw\" (UniqueName: \"kubernetes.io/projected/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-kube-api-access-vb5rw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.183487 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.183310 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.183487 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.183372 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.183687 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.183616 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.183745 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.183729 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.183745 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.183740 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.185853 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.185831 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.185959 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.185941 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.198622 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.198592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb5rw\" (UniqueName: \"kubernetes.io/projected/52d3caf0-1f5d-4ed0-b4a9-b15781f494ba-kube-api-access-vb5rw\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9\" (UID: \"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.349114 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.349019 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:18.515630 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:18.515605 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9"] Jul 10 14:55:18.517371 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:55:18.517344 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52d3caf0_1f5d_4ed0_b4a9_b15781f494ba.slice/crio-1c60f742587eaf30e50354a9138b329ecd1a9a5cd9613391af6c5c297922853f WatchSource:0}: Error finding container 1c60f742587eaf30e50354a9138b329ecd1a9a5cd9613391af6c5c297922853f: Status 404 returned error can't find the container with id 1c60f742587eaf30e50354a9138b329ecd1a9a5cd9613391af6c5c297922853f Jul 10 14:55:19.509642 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:19.509603 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" event={"ID":"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba","Type":"ContainerStarted","Data":"9539acda23c1ebf215feb7dc7f5510c2a21d2c005bde473cde7273ee56feb6ed"} Jul 10 14:55:19.510058 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:19.509652 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" event={"ID":"52d3caf0-1f5d-4ed0-b4a9-b15781f494ba","Type":"ContainerStarted","Data":"1c60f742587eaf30e50354a9138b329ecd1a9a5cd9613391af6c5c297922853f"} Jul 10 14:55:19.529856 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:19.529811 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" podStartSLOduration=1.314340805 podStartE2EDuration="1.529787077s" totalImagesPullingTime="215.446272ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:55:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:55:19.527526783 +0000 UTC m=+599.808041475" watchObservedRunningTime="2026-07-10 14:55:19.529787077 +0000 UTC m=+599.810302219" Jul 10 14:55:20.169529 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:20.169447 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 14:55:20.169529 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:20.169466 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 14:55:25.296695 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:25.296664 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:55:25.527514 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:25.527486 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:25.527692 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:25.527545 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:25.539902 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:25.539876 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:25.556404 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:25.556319 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6skm2" Jul 10 14:55:28.349973 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:28.349935 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:28.350499 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:28.349996 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:28.362788 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:28.362761 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:55:28.574365 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:55:28.574318 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9" Jul 10 14:57:14.106512 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:14.106432 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:57:25.399990 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:25.399953 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:57:37.002176 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:37.002136 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:57:44.552590 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:44.552548 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:57:56.124964 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:56.124931 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:57:58.219454 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:58.219390 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68f867f7b6-b7b47"] Jul 10 14:57:58.220063 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:58.219675 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-68f867f7b6-b7b47" podUID="6a21dacd-b810-45f0-a59f-5ab15645b575" containerName="manager" containerID="cri-o://73f39cf182d9bbcea270e98ca293ae2b6106296f224d012eee87cec1148f04fe" gracePeriod=30 Jul 10 14:57:58.425443 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:58.425404 2562 generic.go:356] "Generic (PLEG): container finished" podID="6a21dacd-b810-45f0-a59f-5ab15645b575" containerID="73f39cf182d9bbcea270e98ca293ae2b6106296f224d012eee87cec1148f04fe" exitCode=0 Jul 10 14:57:58.425610 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:58.425483 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68f867f7b6-b7b47" event={"ID":"6a21dacd-b810-45f0-a59f-5ab15645b575","Type":"ContainerDied","Data":"73f39cf182d9bbcea270e98ca293ae2b6106296f224d012eee87cec1148f04fe"} Jul 10 14:57:58.474009 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:58.473948 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:57:58.571628 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:58.571597 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a21dacd-b810-45f0-a59f-5ab15645b575-webhook-cert\") pod \"6a21dacd-b810-45f0-a59f-5ab15645b575\" (UID: \"6a21dacd-b810-45f0-a59f-5ab15645b575\") " Jul 10 14:57:58.571817 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:58.571699 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r26vl\" (UniqueName: \"kubernetes.io/projected/6a21dacd-b810-45f0-a59f-5ab15645b575-kube-api-access-r26vl\") pod \"6a21dacd-b810-45f0-a59f-5ab15645b575\" (UID: \"6a21dacd-b810-45f0-a59f-5ab15645b575\") " Jul 10 14:57:58.573798 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:58.573764 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a21dacd-b810-45f0-a59f-5ab15645b575-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6a21dacd-b810-45f0-a59f-5ab15645b575" (UID: "6a21dacd-b810-45f0-a59f-5ab15645b575"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:57:58.573922 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:58.573816 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a21dacd-b810-45f0-a59f-5ab15645b575-kube-api-access-r26vl" (OuterVolumeSpecName: "kube-api-access-r26vl") pod "6a21dacd-b810-45f0-a59f-5ab15645b575" (UID: "6a21dacd-b810-45f0-a59f-5ab15645b575"). InnerVolumeSpecName "kube-api-access-r26vl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:57:58.673324 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:58.673284 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a21dacd-b810-45f0-a59f-5ab15645b575-webhook-cert\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:57:58.673324 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:58.673318 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r26vl\" (UniqueName: \"kubernetes.io/projected/6a21dacd-b810-45f0-a59f-5ab15645b575-kube-api-access-r26vl\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 14:57:59.432081 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:59.432054 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68f867f7b6-b7b47" Jul 10 14:57:59.432543 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:59.432050 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68f867f7b6-b7b47" event={"ID":"6a21dacd-b810-45f0-a59f-5ab15645b575","Type":"ContainerDied","Data":"a1caa1744466be80754f6bd686c47febe7c1fbb2781e6a8634e8b45135ed033c"} Jul 10 14:57:59.432543 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:59.432191 2562 scope.go:117] "RemoveContainer" containerID="73f39cf182d9bbcea270e98ca293ae2b6106296f224d012eee87cec1148f04fe" Jul 10 14:57:59.453730 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:59.453700 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68f867f7b6-b7b47"] Jul 10 14:57:59.457581 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:57:59.457557 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68f867f7b6-b7b47"] Jul 10 14:58:00.240863 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.240832 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68f867f7b6-27gzw"] Jul 10 14:58:00.241265 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.241254 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a21dacd-b810-45f0-a59f-5ab15645b575" containerName="manager" Jul 10 14:58:00.241311 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.241267 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a21dacd-b810-45f0-a59f-5ab15645b575" containerName="manager" Jul 10 14:58:00.241383 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.241370 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a21dacd-b810-45f0-a59f-5ab15645b575" containerName="manager" Jul 10 14:58:00.245803 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.245785 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68f867f7b6-27gzw" Jul 10 14:58:00.249717 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.249696 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-scqc9\"" Jul 10 14:58:00.249717 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.249708 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 14:58:00.249886 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.249722 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 14:58:00.254465 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.254447 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68f867f7b6-27gzw"] Jul 10 14:58:00.281303 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.281275 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a21dacd-b810-45f0-a59f-5ab15645b575" path="/var/lib/kubelet/pods/6a21dacd-b810-45f0-a59f-5ab15645b575/volumes" Jul 10 14:58:00.389068 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.389040 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c24117e8-6104-47cb-a910-bdfb6007ea7e-webhook-cert\") pod \"maas-controller-68f867f7b6-27gzw\" (UID: \"c24117e8-6104-47cb-a910-bdfb6007ea7e\") " pod="opendatahub/maas-controller-68f867f7b6-27gzw" Jul 10 14:58:00.389254 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.389151 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gw6x\" (UniqueName: \"kubernetes.io/projected/c24117e8-6104-47cb-a910-bdfb6007ea7e-kube-api-access-4gw6x\") pod \"maas-controller-68f867f7b6-27gzw\" (UID: \"c24117e8-6104-47cb-a910-bdfb6007ea7e\") " pod="opendatahub/maas-controller-68f867f7b6-27gzw" Jul 10 14:58:00.489908 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.489871 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c24117e8-6104-47cb-a910-bdfb6007ea7e-webhook-cert\") pod \"maas-controller-68f867f7b6-27gzw\" (UID: \"c24117e8-6104-47cb-a910-bdfb6007ea7e\") " pod="opendatahub/maas-controller-68f867f7b6-27gzw" Jul 10 14:58:00.490418 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.489952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gw6x\" (UniqueName: \"kubernetes.io/projected/c24117e8-6104-47cb-a910-bdfb6007ea7e-kube-api-access-4gw6x\") pod \"maas-controller-68f867f7b6-27gzw\" (UID: \"c24117e8-6104-47cb-a910-bdfb6007ea7e\") " pod="opendatahub/maas-controller-68f867f7b6-27gzw" Jul 10 14:58:00.492609 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.492554 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c24117e8-6104-47cb-a910-bdfb6007ea7e-webhook-cert\") pod \"maas-controller-68f867f7b6-27gzw\" (UID: \"c24117e8-6104-47cb-a910-bdfb6007ea7e\") " pod="opendatahub/maas-controller-68f867f7b6-27gzw" Jul 10 14:58:00.498095 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.498067 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gw6x\" (UniqueName: \"kubernetes.io/projected/c24117e8-6104-47cb-a910-bdfb6007ea7e-kube-api-access-4gw6x\") pod \"maas-controller-68f867f7b6-27gzw\" (UID: \"c24117e8-6104-47cb-a910-bdfb6007ea7e\") " pod="opendatahub/maas-controller-68f867f7b6-27gzw" Jul 10 14:58:00.558252 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.558214 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68f867f7b6-27gzw" Jul 10 14:58:00.698279 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.698249 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68f867f7b6-27gzw"] Jul 10 14:58:00.700017 ip-10-0-135-1 kubenswrapper[2562]: W0710 14:58:00.699991 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc24117e8_6104_47cb_a910_bdfb6007ea7e.slice/crio-951173c9da34cce5f6592b735bf28064f145ef0adbd090e678f1e36ffd4b7d2c WatchSource:0}: Error finding container 951173c9da34cce5f6592b735bf28064f145ef0adbd090e678f1e36ffd4b7d2c: Status 404 returned error can't find the container with id 951173c9da34cce5f6592b735bf28064f145ef0adbd090e678f1e36ffd4b7d2c Jul 10 14:58:00.701478 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:00.701460 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 14:58:01.446221 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:01.446175 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68f867f7b6-27gzw" event={"ID":"c24117e8-6104-47cb-a910-bdfb6007ea7e","Type":"ContainerStarted","Data":"8015a118099d08bbf0286e1221236e2014d548d2c6c8419520cbde84c5256d80"} Jul 10 14:58:01.446426 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:01.446244 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68f867f7b6-27gzw" event={"ID":"c24117e8-6104-47cb-a910-bdfb6007ea7e","Type":"ContainerStarted","Data":"951173c9da34cce5f6592b735bf28064f145ef0adbd090e678f1e36ffd4b7d2c"} Jul 10 14:58:01.446426 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:01.446275 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68f867f7b6-27gzw" Jul 10 14:58:01.466685 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:01.466635 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68f867f7b6-27gzw" podStartSLOduration=0.943786909 podStartE2EDuration="1.466618849s" totalImagesPullingTime="522.83194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:58:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:58:01.462932085 +0000 UTC m=+761.743446787" watchObservedRunningTime="2026-07-10 14:58:01.466618849 +0000 UTC m=+761.747133547" Jul 10 14:58:12.457800 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:12.457767 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68f867f7b6-27gzw" Jul 10 14:58:23.403278 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:23.403244 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:58:35.906252 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:58:35.906216 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:59:10.690564 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:59:10.690478 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:59:19.906457 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:59:19.906420 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:59:28.910921 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:59:28.910886 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:59:37.813181 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:59:37.813147 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:59:47.641039 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:59:47.641003 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 14:59:57.000194 ip-10-0-135-1 kubenswrapper[2562]: I0710 14:59:57.000149 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:00:09.606407 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:00:09.606365 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:00:19.895374 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:00:19.895326 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:00:20.218198 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:00:20.218122 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:00:20.220244 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:00:20.220219 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:00:41.492852 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:00:41.492811 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:00:52.909473 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:00:52.909440 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:01:15.516099 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:01:15.516064 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:01:28.420765 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:01:28.420733 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:01:43.897962 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:01:43.897879 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:01:53.618195 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:01:53.618162 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:02:03.219938 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:02:03.219889 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:02:12.596614 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:02:12.596582 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:02:21.990846 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:02:21.990811 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:04:12.013753 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:04:12.013710 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:04:20.904535 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:04:20.904501 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:04:36.230257 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:04:36.230225 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:04:45.007888 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:04:45.007795 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:04:56.672289 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:04:56.672252 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:05:07.510771 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:05:07.510733 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:05:16.597297 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:05:16.597264 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:05:20.259012 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:05:20.258982 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:05:20.262555 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:05:20.262526 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:05:27.591643 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:05:27.591608 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:05:38.801346 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:05:38.801305 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:05:47.232595 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:05:47.232560 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:05:57.994536 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:05:57.994503 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:06:19.802879 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:06:19.802802 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:06:29.331227 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:06:29.331189 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:06:38.708272 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:06:38.708241 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:06:50.448772 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:06:50.448735 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:06:58.529393 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:06:58.529358 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:07:10.006633 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:10.006601 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:07:23.610695 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:23.610660 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:07:33.239379 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.239327 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-89f675957-bczvn"] Jul 10 15:07:33.243070 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.243052 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.261983 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.261960 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-89f675957-bczvn"] Jul 10 15:07:33.302864 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.302831 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:07:33.350462 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.350429 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-oauth-serving-cert\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.350608 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.350466 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxnl7\" (UniqueName: \"kubernetes.io/projected/ea5f42cd-6f93-4e86-9569-cdbea1956456-kube-api-access-dxnl7\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.350608 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.350581 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-trusted-ca-bundle\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.350697 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.350616 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-service-ca\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.350697 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.350644 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea5f42cd-6f93-4e86-9569-cdbea1956456-console-serving-cert\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.350766 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.350693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-console-config\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.350766 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.350753 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea5f42cd-6f93-4e86-9569-cdbea1956456-console-oauth-config\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.451726 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.451691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea5f42cd-6f93-4e86-9569-cdbea1956456-console-oauth-config\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.451902 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.451736 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-oauth-serving-cert\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.451902 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.451754 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxnl7\" (UniqueName: \"kubernetes.io/projected/ea5f42cd-6f93-4e86-9569-cdbea1956456-kube-api-access-dxnl7\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.451902 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.451878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-trusted-ca-bundle\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.452024 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.451928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-service-ca\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.452024 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.451978 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea5f42cd-6f93-4e86-9569-cdbea1956456-console-serving-cert\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.452024 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.452012 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-console-config\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.452552 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.452526 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-oauth-serving-cert\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.452677 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.452657 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-service-ca\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.452918 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.452895 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-console-config\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.453002 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.452920 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea5f42cd-6f93-4e86-9569-cdbea1956456-trusted-ca-bundle\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.454428 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.454408 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea5f42cd-6f93-4e86-9569-cdbea1956456-console-oauth-config\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.454536 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.454510 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea5f42cd-6f93-4e86-9569-cdbea1956456-console-serving-cert\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.459303 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.459279 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxnl7\" (UniqueName: \"kubernetes.io/projected/ea5f42cd-6f93-4e86-9569-cdbea1956456-kube-api-access-dxnl7\") pod \"console-89f675957-bczvn\" (UID: \"ea5f42cd-6f93-4e86-9569-cdbea1956456\") " pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.552965 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.552933 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:33.697260 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.697232 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-89f675957-bczvn"] Jul 10 15:07:33.699185 ip-10-0-135-1 kubenswrapper[2562]: W0710 15:07:33.699152 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5f42cd_6f93_4e86_9569_cdbea1956456.slice/crio-c29e17af60e1a3416d5c49f6d4a4c11120c57e80dbbc136eeaae066d99b470c6 WatchSource:0}: Error finding container c29e17af60e1a3416d5c49f6d4a4c11120c57e80dbbc136eeaae066d99b470c6: Status 404 returned error can't find the container with id c29e17af60e1a3416d5c49f6d4a4c11120c57e80dbbc136eeaae066d99b470c6 Jul 10 15:07:33.730422 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:33.730395 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-89f675957-bczvn" event={"ID":"ea5f42cd-6f93-4e86-9569-cdbea1956456","Type":"ContainerStarted","Data":"c29e17af60e1a3416d5c49f6d4a4c11120c57e80dbbc136eeaae066d99b470c6"} Jul 10 15:07:34.737012 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:34.736978 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-89f675957-bczvn" event={"ID":"ea5f42cd-6f93-4e86-9569-cdbea1956456","Type":"ContainerStarted","Data":"1630778c355e12fd05cf1a7787092dc85e82da6e7427f79b0a680f8d5dee8e3c"} Jul 10 15:07:34.757353 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:34.757293 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-89f675957-bczvn" podStartSLOduration=1.757275392 podStartE2EDuration="1.757275392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:07:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:07:34.756750827 +0000 UTC m=+1335.037265525" watchObservedRunningTime="2026-07-10 15:07:34.757275392 +0000 UTC m=+1335.037790083" Jul 10 15:07:43.553942 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:43.553863 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:43.553942 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:43.553918 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:43.558809 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:43.558787 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:43.803136 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:43.803105 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-89f675957-bczvn" Jul 10 15:07:43.852123 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:43.852008 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bb985b848-m4fb7"] Jul 10 15:07:47.468063 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:47.468026 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:07:56.504002 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:07:56.503962 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:08:08.876222 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:08.876181 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bb985b848-m4fb7" podUID="4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" containerName="console" containerID="cri-o://7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441" gracePeriod=15 Jul 10 15:08:09.136907 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.136850 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 15:08:09.184130 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.184103 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-oauth-serving-cert\") pod \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " Jul 10 15:08:09.184130 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.184132 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-oauth-config\") pod \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " Jul 10 15:08:09.184367 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.184155 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2f9g5\" (UniqueName: \"kubernetes.io/projected/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-kube-api-access-2f9g5\") pod \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " Jul 10 15:08:09.184367 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.184181 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-serving-cert\") pod \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " Jul 10 15:08:09.184367 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.184200 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-service-ca\") pod \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " Jul 10 15:08:09.184367 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.184248 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-config\") pod \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " Jul 10 15:08:09.184367 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.184310 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-trusted-ca-bundle\") pod \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\" (UID: \"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d\") " Jul 10 15:08:09.184677 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.184574 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" (UID: "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 15:08:09.184844 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.184820 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-service-ca" (OuterVolumeSpecName: "service-ca") pod "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" (UID: "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 15:08:09.184917 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.184841 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" (UID: "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 15:08:09.184977 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.184894 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-config" (OuterVolumeSpecName: "console-config") pod "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" (UID: "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 15:08:09.186537 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.186516 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" (UID: "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 15:08:09.186643 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.186624 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" (UID: "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 15:08:09.186893 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.186878 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-kube-api-access-2f9g5" (OuterVolumeSpecName: "kube-api-access-2f9g5") pod "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" (UID: "4cdd21dc-d580-4da1-b8bb-cf86dcc5080d"). InnerVolumeSpecName "kube-api-access-2f9g5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:08:09.285996 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.285965 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-serving-cert\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 15:08:09.285996 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.285993 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-service-ca\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 15:08:09.285996 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.286002 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-config\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 15:08:09.286238 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.286010 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-trusted-ca-bundle\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 15:08:09.286238 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.286019 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-oauth-serving-cert\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 15:08:09.286238 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.286028 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-console-oauth-config\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 15:08:09.286238 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.286036 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f9g5\" (UniqueName: \"kubernetes.io/projected/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d-kube-api-access-2f9g5\") on node \"ip-10-0-135-1.ec2.internal\" DevicePath \"\"" Jul 10 15:08:09.950987 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.950951 2562 generic.go:356] "Generic (PLEG): container finished" podID="4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" containerID="7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441" exitCode=0 Jul 10 15:08:09.951395 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.950997 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb985b848-m4fb7" Jul 10 15:08:09.951395 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.951009 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb985b848-m4fb7" event={"ID":"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d","Type":"ContainerDied","Data":"7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441"} Jul 10 15:08:09.951395 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.951056 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb985b848-m4fb7" event={"ID":"4cdd21dc-d580-4da1-b8bb-cf86dcc5080d","Type":"ContainerDied","Data":"822423377d5e35050c04242341260f64b741fb22a15d7d9255ab28aba0e5b9e0"} Jul 10 15:08:09.951395 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.951077 2562 scope.go:117] "RemoveContainer" containerID="7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441" Jul 10 15:08:09.961716 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.961690 2562 scope.go:117] "RemoveContainer" containerID="7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441" Jul 10 15:08:09.962073 ip-10-0-135-1 kubenswrapper[2562]: E0710 15:08:09.962021 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441\": container with ID starting with 7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441 not found: ID does not exist" containerID="7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441" Jul 10 15:08:09.962073 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.962060 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441"} err="failed to get container status \"7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441\": rpc error: code = NotFound desc = could not find container \"7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441\": container with ID starting with 7a9e6fdc2b7f9c08b946c9cd1fcbe3d273ede904389af932e7436f7ca30a8441 not found: ID does not exist" Jul 10 15:08:09.974830 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.974807 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bb985b848-m4fb7"] Jul 10 15:08:09.980971 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:09.980948 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bb985b848-m4fb7"] Jul 10 15:08:10.001921 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:10.001892 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:08:10.280892 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:10.280861 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" path="/var/lib/kubelet/pods/4cdd21dc-d580-4da1-b8bb-cf86dcc5080d/volumes" Jul 10 15:08:19.300076 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:19.300044 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:08:28.600449 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:28.600416 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:08:40.293216 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:40.293179 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:08:52.602349 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:08:52.602305 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:09:01.096527 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:09:01.096487 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:09:13.218145 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:09:13.218050 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:09:25.803169 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:09:25.803133 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:09:38.792089 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:09:38.792052 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:10:20.314063 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:10:20.314032 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:10:20.316545 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:10:20.316526 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:15:20.352360 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:15:20.352310 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:15:20.355552 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:15:20.355531 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:16:37.600482 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:16:37.600446 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:16:49.939643 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:16:49.939609 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:18:09.797462 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:18:09.797425 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:18:17.728804 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:18:17.728730 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:18:26.898116 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:18:26.898078 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:20:20.390442 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:20:20.390413 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:20:20.395420 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:20:20.395396 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:25:20.431084 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:25:20.430977 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:25:20.442431 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:25:20.442403 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:25:27.907395 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:25:27.907352 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:25:42.104258 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:25:42.104172 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mjnkv"] Jul 10 15:29:44.027815 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:44.027785 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-nk2g7_86ba6845-0174-49b1-aed6-0d28e5820ee6/manager/0.log" Jul 10 15:29:44.154750 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:44.154714 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bctfc_374aa998-1266-4f02-a4e2-578cebf7fc1d/manager/0.log" Jul 10 15:29:44.272118 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:44.272082 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-68f867f7b6-27gzw_c24117e8-6104-47cb-a910-bdfb6007ea7e/manager/0.log" Jul 10 15:29:44.393931 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:44.393852 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7zw2b_d0ca89ac-201e-4f30-9a1c-27e7dda107a2/server/0.log" Jul 10 15:29:44.516719 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:44.516691 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c8zx4_8ea08f40-1256-453f-aff3-73749e4262e6/manager/0.log" Jul 10 15:29:44.773466 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:44.773436 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-666496fd47-hjhdd_f9327d34-3718-4635-ae3f-8f7508fb0ee4/copy-manifests/0.log" Jul 10 15:29:44.781233 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:44.781213 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-666496fd47-hjhdd_f9327d34-3718-4635-ae3f-8f7508fb0ee4/manager/0.log" Jul 10 15:29:45.691699 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:45.691667 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d_a6f7912d-a51c-4048-960c-5a72c447d47a/pull/0.log" Jul 10 15:29:45.698062 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:45.698039 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d_a6f7912d-a51c-4048-960c-5a72c447d47a/extract/0.log" Jul 10 15:29:45.704021 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:45.704000 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d_a6f7912d-a51c-4048-960c-5a72c447d47a/util/0.log" Jul 10 15:29:45.819595 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:45.819565 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn_2b8bd9ad-6a93-479d-8505-bf63a4598f3d/util/0.log" Jul 10 15:29:45.826641 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:45.826615 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn_2b8bd9ad-6a93-479d-8505-bf63a4598f3d/pull/0.log" Jul 10 15:29:45.833491 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:45.833466 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn_2b8bd9ad-6a93-479d-8505-bf63a4598f3d/extract/0.log" Jul 10 15:29:45.946085 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:45.946005 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs_77604509-3f54-4ab6-8c8f-7506b471507f/util/0.log" Jul 10 15:29:45.952443 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:45.952424 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs_77604509-3f54-4ab6-8c8f-7506b471507f/pull/0.log" Jul 10 15:29:45.959573 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:45.959545 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs_77604509-3f54-4ab6-8c8f-7506b471507f/extract/0.log" Jul 10 15:29:46.072253 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:46.072220 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2_eb8996d9-2bd4-4342-8649-0a9161859e33/util/0.log" Jul 10 15:29:46.078872 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:46.078846 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2_eb8996d9-2bd4-4342-8649-0a9161859e33/pull/0.log" Jul 10 15:29:46.084939 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:46.084902 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2_eb8996d9-2bd4-4342-8649-0a9161859e33/extract/0.log" Jul 10 15:29:46.315561 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:46.315535 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-82csm_7806869e-eb08-4dd1-ad59-ff3cbb288ddb/manager/0.log" Jul 10 15:29:46.434873 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:46.434838 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-49tqn_dd2650cd-cd7f-44d5-89c2-93c06eb657cb/manager/0.log" Jul 10 15:29:46.546435 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:46.546404 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gzdq6_7fd000f6-c12f-4e15-8141-b7c5d7dcabf8/kuadrant-console-plugin/0.log" Jul 10 15:29:46.663741 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:46.663654 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pcszf_53c37ada-cc91-4749-a306-229570685481/registry-server/0.log" Jul 10 15:29:46.955515 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:46.955439 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-mjnkv_361be177-00db-4509-9ccf-9021f36f06cf/limitador/0.log" Jul 10 15:29:47.073187 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:47.073151 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-smfzf_a25fabad-7c56-4d6f-9a20-0efd4f358d8c/manager/0.log" Jul 10 15:29:47.677040 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:47.677008 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75598f7db4-fdh4p_14830b58-44e5-4e38-b2fa-2ef21983671b/kube-auth-proxy/0.log" Jul 10 15:29:48.019920 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:48.019891 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z4pgz_b758d2e9-5dfd-4889-8700-13e1703ed785/payload-pre-processing/0.log" Jul 10 15:29:48.138067 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:48.138038 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6hs5q_356d1473-1d2c-4ab0-87cc-5b73e768848e/payload-processing/0.log" Jul 10 15:29:48.727873 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:48.727841 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-zfnmd_2bfa1839-0882-4cb7-b409-dfdcc9e369f5/main/0.log" Jul 10 15:29:48.975286 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:48.975244 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhgg87_95ccaef0-b406-4139-891b-afe9d5cc1c88/main/0.log" Jul 10 15:29:49.098945 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:49.098918 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6skm2_67aee487-6e3a-41a9-9cb9-3eae80024733/main/0.log" Jul 10 15:29:49.212546 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:49.212522 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-pm2c9_52d3caf0-1f5d-4ed0-b4a9-b15781f494ba/main/0.log" Jul 10 15:29:56.744412 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:56.744378 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dfm2x_756c9aaf-f794-44f5-98fe-3ba47f92e0dd/global-pull-secret-syncer/0.log" Jul 10 15:29:56.900164 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:56.900104 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-thckg_67ff882f-6e91-4753-889f-2dbabaa9ac18/konnectivity-agent/0.log" Jul 10 15:29:56.938939 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:29:56.938903 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-1.ec2.internal_27fe56af2837f03ffde8b81c782d06c3/haproxy/0.log" Jul 10 15:30:00.833022 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:00.832975 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d_a6f7912d-a51c-4048-960c-5a72c447d47a/extract/0.log" Jul 10 15:30:00.868865 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:00.868831 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d_a6f7912d-a51c-4048-960c-5a72c447d47a/util/0.log" Jul 10 15:30:00.915466 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:00.915428 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hhb5d_a6f7912d-a51c-4048-960c-5a72c447d47a/pull/0.log" Jul 10 15:30:00.966418 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:00.966390 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn_2b8bd9ad-6a93-479d-8505-bf63a4598f3d/extract/0.log" Jul 10 15:30:00.994524 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:00.994499 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn_2b8bd9ad-6a93-479d-8505-bf63a4598f3d/util/0.log" Jul 10 15:30:01.019802 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.019774 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bbcgn_2b8bd9ad-6a93-479d-8505-bf63a4598f3d/pull/0.log" Jul 10 15:30:01.043995 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.043967 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs_77604509-3f54-4ab6-8c8f-7506b471507f/extract/0.log" Jul 10 15:30:01.063295 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.063267 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs_77604509-3f54-4ab6-8c8f-7506b471507f/util/0.log" Jul 10 15:30:01.082641 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.082618 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zzprs_77604509-3f54-4ab6-8c8f-7506b471507f/pull/0.log" Jul 10 15:30:01.108254 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.108182 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2_eb8996d9-2bd4-4342-8649-0a9161859e33/extract/0.log" Jul 10 15:30:01.127488 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.127468 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2_eb8996d9-2bd4-4342-8649-0a9161859e33/util/0.log" Jul 10 15:30:01.150128 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.150108 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19dpg2_eb8996d9-2bd4-4342-8649-0a9161859e33/pull/0.log" Jul 10 15:30:01.541836 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.541800 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-82csm_7806869e-eb08-4dd1-ad59-ff3cbb288ddb/manager/0.log" Jul 10 15:30:01.566573 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.566547 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-49tqn_dd2650cd-cd7f-44d5-89c2-93c06eb657cb/manager/0.log" Jul 10 15:30:01.603223 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.603196 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gzdq6_7fd000f6-c12f-4e15-8141-b7c5d7dcabf8/kuadrant-console-plugin/0.log" Jul 10 15:30:01.641035 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.641004 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pcszf_53c37ada-cc91-4749-a306-229570685481/registry-server/0.log" Jul 10 15:30:01.820298 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.820213 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-mjnkv_361be177-00db-4509-9ccf-9021f36f06cf/limitador/0.log" Jul 10 15:30:01.874746 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:01.874716 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-smfzf_a25fabad-7c56-4d6f-9a20-0efd4f358d8c/manager/0.log" Jul 10 15:30:03.441613 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.441583 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7573c7b0-4f6b-44aa-8cba-a05da8df9fff/alertmanager/0.log" Jul 10 15:30:03.465444 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.465418 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7573c7b0-4f6b-44aa-8cba-a05da8df9fff/config-reloader/0.log" Jul 10 15:30:03.486891 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.486841 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7573c7b0-4f6b-44aa-8cba-a05da8df9fff/kube-rbac-proxy-web/0.log" Jul 10 15:30:03.508155 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.508125 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7573c7b0-4f6b-44aa-8cba-a05da8df9fff/kube-rbac-proxy/0.log" Jul 10 15:30:03.530557 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.530529 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7573c7b0-4f6b-44aa-8cba-a05da8df9fff/kube-rbac-proxy-metric/0.log" Jul 10 15:30:03.551061 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.551031 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7573c7b0-4f6b-44aa-8cba-a05da8df9fff/prom-label-proxy/0.log" Jul 10 15:30:03.575169 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.575144 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7573c7b0-4f6b-44aa-8cba-a05da8df9fff/init-config-reloader/0.log" Jul 10 15:30:03.644449 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.644402 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-shfgb_c2335d6c-8954-4974-978e-3a28da1d1610/kube-state-metrics/0.log" Jul 10 15:30:03.670587 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.670561 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-shfgb_c2335d6c-8954-4974-978e-3a28da1d1610/kube-rbac-proxy-main/0.log" Jul 10 15:30:03.697186 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.697115 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-shfgb_c2335d6c-8954-4974-978e-3a28da1d1610/kube-rbac-proxy-self/0.log" Jul 10 15:30:03.743594 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.743563 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5bdbb7ccff-ns9qg_17cacce8-2719-4477-9f5f-16d924a2b1f3/metrics-server/0.log" Jul 10 15:30:03.900908 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.900887 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7wwz2_5adbec70-77f2-46b8-bbd1-d28b1f08b280/node-exporter/0.log" Jul 10 15:30:03.921492 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.921472 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7wwz2_5adbec70-77f2-46b8-bbd1-d28b1f08b280/kube-rbac-proxy/0.log" Jul 10 15:30:03.952022 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:03.951941 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7wwz2_5adbec70-77f2-46b8-bbd1-d28b1f08b280/init-textfile/0.log" Jul 10 15:30:04.053637 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:04.053598 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-djt7p_ef251af1-dc02-4555-9c6f-1f3b08fd2174/kube-rbac-proxy-main/0.log" Jul 10 15:30:04.079665 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:04.079638 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-djt7p_ef251af1-dc02-4555-9c6f-1f3b08fd2174/kube-rbac-proxy-self/0.log" Jul 10 15:30:04.101666 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:04.101640 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-djt7p_ef251af1-dc02-4555-9c6f-1f3b08fd2174/openshift-state-metrics/0.log" Jul 10 15:30:04.323079 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:04.323053 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-jmpjq_fd0267ce-450e-4a55-b7fa-362e7d8dfc2e/prometheus-operator/0.log" Jul 10 15:30:04.343452 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:04.343419 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-jmpjq_fd0267ce-450e-4a55-b7fa-362e7d8dfc2e/kube-rbac-proxy/0.log" Jul 10 15:30:06.710168 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.710133 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k"] Jul 10 15:30:06.710643 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.710628 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" containerName="console" Jul 10 15:30:06.710643 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.710645 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" containerName="console" Jul 10 15:30:06.710741 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.710731 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cdd21dc-d580-4da1-b8bb-cf86dcc5080d" containerName="console" Jul 10 15:30:06.714250 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.714234 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.716861 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.716836 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bc474\"/\"default-dockercfg-4gfb9\"" Jul 10 15:30:06.716861 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.716849 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bc474\"/\"kube-root-ca.crt\"" Jul 10 15:30:06.717829 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.717805 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bc474\"/\"openshift-service-ca.crt\"" Jul 10 15:30:06.721548 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.721526 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k"] Jul 10 15:30:06.768459 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.768424 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzwdb\" (UniqueName: \"kubernetes.io/projected/11eb71ec-280f-4723-b428-9337e16f4c8d-kube-api-access-kzwdb\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.768459 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.768459 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-lib-modules\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.768720 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.768486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-podres\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.768720 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.768561 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-proc\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.768720 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.768640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-sys\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.869679 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.869636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzwdb\" (UniqueName: \"kubernetes.io/projected/11eb71ec-280f-4723-b428-9337e16f4c8d-kube-api-access-kzwdb\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.869873 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.869687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-lib-modules\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.869873 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.869738 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-podres\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.869873 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.869800 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-proc\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.869873 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.869856 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-sys\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.870063 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.869892 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-lib-modules\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.870063 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.869928 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-podres\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.870063 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.869928 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-proc\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.870063 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.869977 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11eb71ec-280f-4723-b428-9337e16f4c8d-sys\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:06.877502 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:06.877479 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzwdb\" (UniqueName: \"kubernetes.io/projected/11eb71ec-280f-4723-b428-9337e16f4c8d-kube-api-access-kzwdb\") pod \"perf-node-gather-daemonset-5qf6k\" (UID: \"11eb71ec-280f-4723-b428-9337e16f4c8d\") " pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:07.026504 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:07.026470 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:07.052132 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:07.052109 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-89f675957-bczvn_ea5f42cd-6f93-4e86-9569-cdbea1956456/console/0.log" Jul 10 15:30:07.090253 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:07.090220 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-gmsr4_1d635b58-4dac-46d2-8398-e652e623cf6c/download-server/0.log" Jul 10 15:30:07.170424 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:07.170400 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k"] Jul 10 15:30:07.172091 ip-10-0-135-1 kubenswrapper[2562]: W0710 15:30:07.172059 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod11eb71ec_280f_4723_b428_9337e16f4c8d.slice/crio-2ceb322769b409eda62aeb0770d73cb1bd9a2bc551af5ca5c1136be03a458b46 WatchSource:0}: Error finding container 2ceb322769b409eda62aeb0770d73cb1bd9a2bc551af5ca5c1136be03a458b46: Status 404 returned error can't find the container with id 2ceb322769b409eda62aeb0770d73cb1bd9a2bc551af5ca5c1136be03a458b46 Jul 10 15:30:07.511997 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:07.511958 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" event={"ID":"11eb71ec-280f-4723-b428-9337e16f4c8d","Type":"ContainerStarted","Data":"180016e0d1d956db6a31412e9b2f1c7b694363acbbef1ef9308a53570a7d2ad9"} Jul 10 15:30:07.512191 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:07.512022 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" event={"ID":"11eb71ec-280f-4723-b428-9337e16f4c8d","Type":"ContainerStarted","Data":"2ceb322769b409eda62aeb0770d73cb1bd9a2bc551af5ca5c1136be03a458b46"} Jul 10 15:30:07.512191 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:07.512153 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:07.530611 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:07.530554 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" podStartSLOduration=1.53053829 podStartE2EDuration="1.53053829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:30:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:30:07.527955547 +0000 UTC m=+2687.808470246" watchObservedRunningTime="2026-07-10 15:30:07.53053829 +0000 UTC m=+2687.811052987" Jul 10 15:30:08.772026 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:08.772000 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-znj5q_475e4f82-6ecd-47d9-88e1-6901ff54ba92/dns/0.log" Jul 10 15:30:08.792084 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:08.792057 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-znj5q_475e4f82-6ecd-47d9-88e1-6901ff54ba92/kube-rbac-proxy/0.log" Jul 10 15:30:08.817710 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:08.817681 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jgcmn_11d98938-d5e6-4a92-ac4b-fa8a333a09d5/dns-node-resolver/0.log" Jul 10 15:30:09.435944 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:09.435916 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkvd4_5d82d7bc-fae9-4db0-941a-6f9988bfe13f/node-ca/0.log" Jul 10 15:30:10.413314 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:10.413238 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75598f7db4-fdh4p_14830b58-44e5-4e38-b2fa-2ef21983671b/kube-auth-proxy/0.log" Jul 10 15:30:10.540346 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:10.540315 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z4pgz_b758d2e9-5dfd-4889-8700-13e1703ed785/payload-pre-processing/0.log" Jul 10 15:30:10.584053 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:10.584021 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6hs5q_356d1473-1d2c-4ab0-87cc-5b73e768848e/payload-processing/0.log" Jul 10 15:30:11.260059 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:11.260031 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wp7kd_e5c6f071-e790-4ccf-909f-131a20bff3ec/serve-healthcheck-canary/0.log" Jul 10 15:30:11.802462 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:11.802433 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hhcdc_fd9bc4d0-177d-465c-8b8f-60daebc95bc6/kube-rbac-proxy/0.log" Jul 10 15:30:11.823626 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:11.823602 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hhcdc_fd9bc4d0-177d-465c-8b8f-60daebc95bc6/exporter/0.log" Jul 10 15:30:11.846604 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:11.846577 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hhcdc_fd9bc4d0-177d-465c-8b8f-60daebc95bc6/extractor/0.log" Jul 10 15:30:13.528887 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:13.528860 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bc474/perf-node-gather-daemonset-5qf6k" Jul 10 15:30:13.964240 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:13.964153 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-nk2g7_86ba6845-0174-49b1-aed6-0d28e5820ee6/manager/0.log" Jul 10 15:30:14.037599 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:14.037565 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bctfc_374aa998-1266-4f02-a4e2-578cebf7fc1d/manager/0.log" Jul 10 15:30:14.108181 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:14.108147 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-68f867f7b6-27gzw_c24117e8-6104-47cb-a910-bdfb6007ea7e/manager/0.log" Jul 10 15:30:14.131278 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:14.131256 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7zw2b_d0ca89ac-201e-4f30-9a1c-27e7dda107a2/server/0.log" Jul 10 15:30:14.185767 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:14.185732 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c8zx4_8ea08f40-1256-453f-aff3-73749e4262e6/manager/0.log" Jul 10 15:30:14.269127 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:14.269101 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-666496fd47-hjhdd_f9327d34-3718-4635-ae3f-8f7508fb0ee4/manager/0.log" Jul 10 15:30:14.292410 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:14.292385 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-666496fd47-hjhdd_f9327d34-3718-4635-ae3f-8f7508fb0ee4/copy-manifests/0.log" Jul 10 15:30:16.535304 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:16.535277 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75598f7db4-fdh4p_14830b58-44e5-4e38-b2fa-2ef21983671b/kube-auth-proxy/0.log" Jul 10 15:30:16.641209 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:16.641175 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z4pgz_b758d2e9-5dfd-4889-8700-13e1703ed785/payload-pre-processing/0.log" Jul 10 15:30:16.677680 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:16.677587 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6hs5q_356d1473-1d2c-4ab0-87cc-5b73e768848e/payload-processing/0.log" Jul 10 15:30:18.090009 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:18.089987 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75598f7db4-fdh4p_14830b58-44e5-4e38-b2fa-2ef21983671b/kube-auth-proxy/0.log" Jul 10 15:30:18.187979 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:18.187946 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z4pgz_b758d2e9-5dfd-4889-8700-13e1703ed785/payload-pre-processing/0.log" Jul 10 15:30:18.222269 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:18.222234 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6hs5q_356d1473-1d2c-4ab0-87cc-5b73e768848e/payload-processing/0.log" Jul 10 15:30:20.478898 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:20.478784 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:30:20.491245 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:20.491221 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:30:22.286910 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:22.286881 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58qv9_1e430080-473f-49a1-8290-dc04122b72c5/kube-multus-additional-cni-plugins/0.log" Jul 10 15:30:22.307973 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:22.307944 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58qv9_1e430080-473f-49a1-8290-dc04122b72c5/egress-router-binary-copy/0.log" Jul 10 15:30:22.328978 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:22.328930 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58qv9_1e430080-473f-49a1-8290-dc04122b72c5/cni-plugins/0.log" Jul 10 15:30:22.349664 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:22.349639 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58qv9_1e430080-473f-49a1-8290-dc04122b72c5/bond-cni-plugin/0.log" Jul 10 15:30:22.370767 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:22.370736 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58qv9_1e430080-473f-49a1-8290-dc04122b72c5/routeoverride-cni/0.log" Jul 10 15:30:22.392283 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:22.392261 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58qv9_1e430080-473f-49a1-8290-dc04122b72c5/whereabouts-cni-bincopy/0.log" Jul 10 15:30:22.412698 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:22.412675 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58qv9_1e430080-473f-49a1-8290-dc04122b72c5/whereabouts-cni/0.log" Jul 10 15:30:22.610125 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:22.610047 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lsgpm_c83b077a-d8b6-4ba6-b5e3-c0b719e2cd76/kube-multus/0.log" Jul 10 15:30:22.796776 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:22.796744 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rr8m4_47c939b7-87e0-4cc4-a214-d47ce0fb2ee2/network-metrics-daemon/0.log" Jul 10 15:30:22.815693 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:22.815658 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rr8m4_47c939b7-87e0-4cc4-a214-d47ce0fb2ee2/kube-rbac-proxy/0.log" Jul 10 15:30:24.010191 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:24.010161 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-controller/0.log" Jul 10 15:30:24.025448 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:24.025427 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/0.log" Jul 10 15:30:24.025636 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:24.025596 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovn-acl-logging/1.log" Jul 10 15:30:24.044393 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:24.044366 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/kube-rbac-proxy-node/0.log" Jul 10 15:30:24.064009 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:24.063984 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/kube-rbac-proxy-ovn-metrics/0.log" Jul 10 15:30:24.085146 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:24.085126 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/northd/0.log" Jul 10 15:30:24.105184 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:24.105161 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/nbdb/0.log" Jul 10 15:30:24.125155 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:24.125131 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/sbdb/0.log" Jul 10 15:30:24.176377 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:24.176322 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zqbmp_e32e12c7-6c2e-455d-b703-6f648413a630/ovnkube-controller/0.log" Jul 10 15:30:25.322664 ip-10-0-135-1 kubenswrapper[2562]: I0710 15:30:25.322633 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-g2z2c_cb1e9cb5-c361-4628-afd2-b341d97047a3/network-check-target-container/0.log"